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I.  INTRODUCTION 


This  thesis  deals  with  detection  and  estimation  using  spatially  separated  sensors. 
A  typical  practical  situation  is  a  surveillance  system  [1]  in  which  a  large  number  of 
sensors  monitor  some  region  of  space,  earth  or  sea  and  report  their  findings  to  a  global 
processor.  The  sensors  themselves  may  use  thermal,  acoustic  or  infrared  effects  to  form 
their  observations.  The  global  processor  performs  some  processing  on  the  data  to  come 
with  a  decision  or  for  taking  actions.  Because  of  many  considerations  such  as 
bandwidth  communication  limitations,  time  delay  or  because  the  amount  of 
information  is  too  massive  to  be  processed  by  a  single  processor,  the  processing  is 
carried  out  on  many  levels.  As  an  example  consider  the  case  of  distributed  detection. 
Detection  is  performed  at  the  sensor  level  and  at  the  fusion  center. 

Due  to  the  loss  of  information  in  the  local  processing,  the  overall  performance 
degrades.  However  a  great  communication  bandwidth  reduction  results.  If  the 
communication  channels  can  support  more  information  flow,  then  it  is  wise  to  perform 
“softer"  processing  at  the  local  level,  to  send  more  information  to  the  fusion  center,  and 
to  use  the  information  available  there  effectively. 

The  purpose  of  this  chapter  is  to  define  the  Distributed  Signal  Processing  (DSP) 
problem  in  general  and  to  show  some  reasons  and  situations  in  which  it  replaces 
Centralized  Signal  Processing  (CSP)  techniques.  We  then  will  review  the  status  of  the 
research  on  Decentralized  Detection  (DD)  problem,  one  of  the  basic  problems  of  DSP. 
Finally  the  contributions  and  organization  of  this  thesis  are  described. 

A.  OVERVIEW 

Classical  (Centralized)  Signal  Processing  (CSP)  assumes  complete  availability  of 
all  information  (signals)  at  one  central  processor  for  processing  (decision  making, 
computing,  detection,  estimation,  etc...).  While  this  situation  is  realistic  in  some  cases, 
many  real  world  systems  are  too  large  for  the  classical  processing  to  be  practically 
applied.  Power  systems,  detection  networks,  large  manufacturing  systems  and  military 
organizations  are  among  those  systems  in  which  total  centralized  signal  processing  is 
hard  to  apply.  Some  of  the  reasons  and  considerations  for  the  limations  of  CSP  are 
(2,3]: 


m 


1.  In  large  systems,  each  processor  has  partial  information  of  some  credibility. 
While  total  information  is  distributed  in  the  whole  system,  total  centeralization 
of  the  information  at  one  processor  is  impractical,  inconvenient  or  expensive 
due  to  limitations  in  the  system's  communication  channels,  memory  or 
computation  and  information  capabilities. 

2.  In  some  cases,  processing  speed  is  a  bottleneck.  Increasing  local  processing  of 
the  data  at  each  processor  and  sending  processed  data  to  the  next  level  of 
processors  will  help  relieve  the  problem. 

3.  When  reliability  of  the  system  is  of  major  concern,  distributed  processing  may 
better  tolerate  various  kinds  of  equipment  failures.  Less  complex  centralized 
processing  is  more  easily  shifted  to  a  new  location. 

4.  In  cases  when  security  is  a  major  problem,  increasing  local  processing  will 
decrease  the  information  handled  between  the  processors,  so  limit  any  other 
system's  access  to  the  process. 

5.  As  the  cost  of  computation  has  decreased  dramatically  relative  to  the  cost  of 
communication,  it  is  advantageous  to  trade  off  increased  computation  for 
reduced  communication.  So  in  Distributed  Sensor  Networks  (DSN)  involving 
geographically  distributed  sensors  that  collect  data,  it  may  be  more  economical 
to  locally  process  the  data  and  send  condensed  summaries  to  other  processors. 

Distributed  Signal  Processing  (DSP),  in  contrast  to  CSP,  has  several  processors 
that  cooperate  together  to  best  achieve  a  global  task  according  to  some  criterion.  A 
basic  problem  in  DSP,  which  has  attracted  much  attention  recently,  is  the 
Decentralized  Detection  (DD)  problem  (hypothesis  testing).  The  DD  problem  will  be 
a  major  concern  in  this  thesis.  A  summary  of  its  status  is  given  in  the  following  section. 

B.  MOTIVATION 

There  has  been  an  increased  interest  in  the  DD  problem  since  Tenney  and 
Sandell  introduced  it  in  1981  [5].  They  extended  the  classical  Bayesian  formulation  of 
the  detection  problem  to  distributed  environments.  Because  their  work  was  the 
pioneering  one  in  DD  and  because  we  will  refer  to  it  often  in  this  thesis  ,  let  us 
consider  it  now  in  some  detail  together  with  the  Centralized  Detection  (CD)  problem. 
Also,  because  detection  is  dealt  with  throughout  a  large  portion  of  this  thesis,  we  will 
make  some  remarks  about  the  phenomena  to  be  detected  and  about  detection  criterion. 
The  Phenomena 

Consider  observing  a  phenomena  H  of  M  possible  states  in  order  to  determine 
which  of  them  is  true.  For  M  =  2,  the  state  HQ  is  called  the  null  hypothesis  and  Hj 
the  alternative  hypothesis.  Their  probabilities  of  occurrence 
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are  assumed  to  be  known. 

The  Sensor  Observations 

The  phenomena  H  is  observed  by  N  sensors  Sj,S2,...,SN  .  The  sensor 

observations  are  ylty2 . ,yN  .  The  sensor  observations  have  known  conditional 

distributions 

p(yi.y2.~.yN/H0),  viyvy2 . ?n  /Hi  )•  (i-2) 

Detection  Criterion 

The  function  of  the  detection  process  is  to  make  a  decision,  UQ,  about  which 
state  of  the  phenomena  is  true.  The  optimality  criterion  is  a  function 


J:  U0  xH  -» 


that  assigns  to  the  event  of  deciding  u-  when  Hj  is  true  a  real  number  C^,  i,j  =  0,l. 
called  the  detection  cost,  so 


J(Uo=Ui  ,H  =  Hj  )=  Cjj. 


The  objective  of  the  decision  rule  will  be  to  minimize  the  expected  decision  cost 


min  E{J(u,H)}. 


An  important  ratio  in  our  analysis  is  the  constant  given  by 


Pp  (Cio'Cqq) 
P1  (C01*Cll) 


Van  Trees  [6]  showed  that  the  average  decision  cost  is  given  by, 


^  ^oo  po  +coi  pi  +po  (C0i  *^n  )  pr  'pi  (C0i  '^n  )Pd 


jjfr-xr 


w 


where  Pf  and  Pd  are  the  probability  of  false  alarm1  and  probability  of  detection2 
respectively.  At  this  point  we  will  make  the  assumptions  that 


C01  >  Cu.  (1.8) 

and 

Cio  51  ^-00  ' 


These  assumptions  implies  that  it  is  more  costly  to  err  than  to  make  a  correct  decision. 
Equation  ( 1.7)  can  then  be  written  in  the  form: 


^on  Pp  +Coi  Pi  +  Pq  (C|0  -C00  )  p 
_  Pi  <coi  *Cn  )  .  Pi  <Coi  *^n  )  f  d_ 


Pi  <Co,  'Cn  )  • 


(1.10) 


Ignoring  positive  constants  that  will  not  affect  our  analysis,  the  average  decision  cost 
R  is  given  by 


R  =  1  +  C  Pf  -  Pd  . 


(Ml) 


1.  The  Centralized  Detection  (CD)  Problem 

The  problem  of  centralized  binary  hypothesis  testing  can  be  posed  in  its  most 
general  form  as  follows.  For  the  structure  of  Figure  1.1  it  is  assumed  that  all  sensor 
observations  can  be  sent  to  one  (central)  location  for  processing.  The  function  of  the 
processor  is  to  map  the  vector  Y=  [  Vj  y2  ...  yN  ]l  into  the  decision  space  U&subo(0,l) 

U0:Y-M°,1)  (M2) 

as  follows; 


C  0,  H0  is  declared  to  have  been  detected 
1,  Hj  is  declared  to  have  been  detected. 


1  Probability  of  deciding  UQ  =0  while  Hj  is  true 
Probability  of  deciding  L'0.=  1  while  is  true 
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(1.13) 


m 


Solution  of  the  CD  Problem 

The  solution  to  the  CD  problem  is  [6] 

a)  deterministic,  so  that  the  decision  rule  is  a  function  of  the  observations 


y  :  Yj  xY2  x...xYn  -♦  (0,1), 


(1.14) 


b)  a  likelihood  ratio  test, 


TT  ,  ,  f  °.  if  a  (ylfy2..,yN  )  *  t 

U°(,1'Tl’  *)_  U  if  a  (y,.y2 . yN)  <  < 


(1.15) 


where 


A  (y1,y2,-.,yN)  = 


flyPy2 . >\  /Ht) 

. Tn  /h0) 


(1.16) 


c)  and  the  threshold  t  is  given  by 


t  =  C 


(1.17) 


2.  The  Decentralized  Detection  (DD)  Problem  with  Fusion 

Consider  the  structure  of  Figure  1.2  with  H  and  Y  being  as  before;  the 
decisions  Uj  ,U2  ,...  and  UN  are  sent  to  a  fusion  center.  The  activity  of  the  fusion 
center  is  to  make  the  global  decision  L"0  according  to  some  preset  fusion  rule. 

U0  :  Uj  xU2..xUN  -»  (0,1).  (1.18) 

In  the  DD  problem  with  fusion  it  is  required  to  design  local  decision  rules  U^Uj,... 
and  UN  and  a  global  fusion  rule  (1.18)  so  as  to  minimize  the  expected  cost  E{J(L'o  ,H)} 
incurred  by  deciding  UQ  =  i  when  FT  is  true. 

Choosing  an  AND  fusion  rule  apriori,  Tenney  and  Sandell  solved  this  problem 
for  N’=2.  They  set  the  decision  rule  as  UQ  =Uj  L'2  and  optimized  the  local  decision 
rules. 


Figure  1.2  Decentralized  L  section  with  Fusion. 
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Solution  of  DD  Problem  with  Fusion 

The  solution  to  the  DD  problem  with  fusion  is 
1*  deterministic 


Yl  •  Yj  -+  (0,1)  (1.19) 

and 


Y2:Y2-*(0.1) 


(1.20) 


2.  a  likelihood  ratio  test  for  each  detector 


where 


tf'Vyo*  li 
ifAi(yi)<  h 


A;  (y,)  - 


flYi  /Ht  ) 

f&i  ;H0  ) 


3.  with  coupled  thresholds  tj  and  t2  given  by 


,  -c 

1  p«d2  ;y, ) 


and 


^  Pr(D1  ;y2  ) 


(1.21) 


(1-22) 


(1.23) 


(1.24) 


where  Pr^  / y.  )  and  Pr(Dj  /y.  )  are  respectively  the  conditional  probability  of 
false  alarm  and  the  conditional  probability  of  detection  of  the  i^  detector  given 
the  jj.  detector's  observation. 


Equations  (1.23  )  and  (1.24  )  are  two  coupled  functional  equations  in  t1  and  t2  .  For 
general  distributions,  a  functional  expression  for  each  of  them  in  terms  of  its  own 
observation  and  the  other  detector's  decision  is  impossible.  We  shall  consider  the 
complexity  of  these  decision  rules  later.  A  special  case  of  the  DD  problem  is  the  case 
of  conditionally  independent  sensor  observations,  i.e. 


f(y1/y2.H)=fiy1  /H) 


(1.25) 


fly2/yi  ,H)  =  f{y2/H). 


In  this  case,  the  conditional  probabilities  in  (1.23  )  and  (1.24)  reduce  to 


‘l  “  C 


(1.26) 


(1.27) 


t2  =  C 


(1-28) 


Equations  (1.27)  and  (1.28)  are  two  coupled  algebraic  equations  in  the  form  of 


t1=g1(t2)  and  t2=g2(tj) 


(1-29) 


since  Pfi  and  Pdi  depend  on  t;  .  This  coupling  represents  cooperation  between  the  two 
sensors.  The  threshold  equations  are  necessary  conditions  for  optimality.  There  may 
be  several  local  minima;  each  must  be  checked  to  assure  the  global  minima.  The 
threshold  equations  are  strongly  coupled  for  general  cost  assumptions. 

Tenney  and  Sandell  came  to  the  following  conclusions: 

1.  Increasing  the  signal-to-noise  ratio  improves  the  performance  of  the  system. 
However  a  centralized  system  makes  more  efficient  use  of  the  increased 
information. 


2.  As  the  imbalance  between  the  two  detectors  increases  the  performance 
improves.  If  the  signal*to-noise  ratio  of  one  of  the  detectors  goes  to  zero  then 
the  system  decision  is  that  of  the  other  detector.  This  is  equivalent  to  the 
performance  of  a  CD  system  of  the  same  signal-to-noise  ratio. 

The  case  of  conditionally  independent  observations  has  been  considered  by 
many  authors.  Sarma  and  Rao  [7]  extended  Tenney  and  Sandell's  results  to  the  case  of 
three  sensors.  They  assumed  a  majority  logic  fusion  rule  and  evaluated  the  threshold 
settings  for  some  specific  cases.  Chair  and  Varshney  [8]  considered  the  problem  of 
optimal  fusion  of  N  local  decisions  from  prespecified  local  decision  rules.  Their 
optimum  fusion  structure  is  a  weighted  sum  of  local  decisions  according  to  their 
reliabilities.  Reibman  and  Nolte  [9]  optimize  both  local  decision  rules  and  the  fusion 
rule  under  the  assumption  of  identical  local  decision  rules.  The  global  decision  is  then  k 
out  of  N.  They  optimize  the  local  decision  rule  for  each  k  ,k=  1,2,...,N,  then  pick  the 
value  of  k  corresponding  to  the  minimum  decision  cost. 

A  sub-class  of  the  DD  problem  with  fusion,  that  will  be  referrd  to  as  the 
"Second  Opinion"  problem,  is  the  fusion  of  one's  observation  with  another's  decisions. 
An  example  of  this  is  the  second  opinion  in  a  medical  examination,  or  even  asking  for 
legal  advise.  Ekchian  [10]  and  Ekchian  and  Tenney  [11]  consider  some  specific 
topologies  of  this  problem.  Each  decision  maker  has  to  make  his  decision  based  on  his 
own  observation  and  a  predecessor's  decisions.  All  the  decision  rules  are  likelihood 
ratio  tests  using  the  actual  data.  The  thresholds  are  determined  by  incoming 
communication  messages.  The  number  of  thresholds  at  each  decision  maker  grows 
exponentially  with  the  number  of  message  inputs.  Their  results  suggest  putting  the 
noisy  sensor  "up  stream"  in  the  detection  network. 

Papastavrou  and  Athans  [12]  also  consider  the  second  opinion  problem.They 
examine  the  structure  of  a  primary  decision  maker,  PDM,  and  a  secondary  decision 
maker,  SDM  (  a  consultant  ).  The  PDM  makes  his  decision  based  on  his  own 
observation  if  it  is  of  good  quality.  If  his  observation  is  noisy,  the  PDM  asks,  at  a 
communication  cost,  the  opinion  of  the  SDM.  Being  activated  by  the  request  of  the 
PDM,  the  SDM  sends  his  decision  to  the  PDM  or  ignores  the  request  if  his 
observation  is  noisy.  In  cither  case  the  PDM  has  to  make  a  final  decision.  Again  the 
thresholds  are  coupled.  The  threshold  of  the  PDM  is  determined  by  the  message  of  the 
SDM. 

This  thesis  is  motivated  mainly  by  three  of  the  above  works  namely; 

1.  Bayesian  formulation  of  the  DD  problem  by  Tenney  and  Sandell  [5]. 


2.  Extension  of  the  DD  problem  to  the  Distributed  Detection  Networks  by 
Ekchian.  [10] 

3.  Extention  of  the  DD  problem  to  the  case  of  correlated  sensor  observations  by 
Lauer  and  Sandell  [4]. 

C.  THE  COMPLEXITY  OF  THE  DD  PROBLEM 

We  saw  that  the  DD  problem  can  be  solved  optimally  for  conditionally 
independent  sensor  observations.  If  this  condition  does  not  hold  local  decisions  are  not 
likelihood  ratio  tests  with  constant  thresholds.  Tenney  and  Sandell  show  that  for 
conditionally  dependent  observations,  local  decision  rules  are  likelihood  ratio  tests  but 
with  data  dependent  thresholds  (see  e.g.  (1.23  )  and  (1.24  )).  These  two  equations  are 
coupled.  This  means  that  the  observation  of  one  sensor  is  necessary  for  the  other 
sensor's  decision,  which  contradicts  the  principle  of  decentralization.  In  terms  of  the 
terminology  of  the  Theory  of  Combinatorial  Complexity  [13],  Tsitisiklis  and  Athans 
[14]  show  that 

1.  The  DD  problem  with  independent  observations  is  a  polynomial  time  problem. 

2.  The  DD  problem  with  dependent  sensor  observations  in  its  simplest  form  is  a 
nondeterministic  polynomial  NP-complete.  This  means  that  exhaustive 
enumeration  is  necessary  to  find  the  optimum  local  decision  rules.  Optimality 
may  be  an  illusive  goal.  So,  suboptimal  solutions  must  be  sought. 

A  suboptimal  solution  to  the  problem  for  the  case  of  AND  fusion  was  considered 
by  Lauer  and  Sandell  [4].  They  considered  the  case  of  known  signals  in  correlated 
noise  They  took  as  a  suboptimal  solution  local  decision  rules  which  are  likelihood 
tests  but  having  constant,  not  data  dependent,  thresholds  satisfying  the  necessary- 
condition  of  optimality.  These  thresholds  are  given  by  the  implicit  equations: 


Al(Tl)=  C 


Pr(F,  ,T ,  ) 
Pr(D2  Tl  ) 


(1-30) 


A,  (T,  )=  C 


Pr(F)  T2  ) 
Pr(Dj  ;T2  ) 


(1.31) 


D.  CONTRIBUTIONS  OF  THIS  THESIS 

We  have  reviewed  the  complexity  of  the  DD  problem  and  its  current  status.  The 
research  reported  here  has  significantly  advanced  this  status  in  several  important  ways. 
Specifically  the  contributions  of  this  thesis  have  been  to  : 

1.  Answer  the  question  of  the  optimum  fusion  rule  at  the  fusion  center  for  the 
case  of  two  sensors. 

2.  Specify  the  exact  relation  between  the  performance  of  the  optimum  fusion  rule 
and  the  correlation  coefficient  between  sensor  observations. 

3.  Solve  the  the  second  opinion  decision  problem. 

4.  Solve  the  multi-level  DD  problem  with  fusion;  i.e.  detection  with  quantized 
sensor  data  for  the  known  signal  in  noise  case. 

5.  Introduce  the  minimum  risk  quantizer. 

6.  Grade  the  road  between  DD  detection  and  CD  detection. 

7.  Optimally  design  quantizers  for  minimum  mean  square  estimation. 

8.  Present  an  efficient  procedure  to  calculate  parameters  of  a  large  variety  of 
quantizers. 

E.  ORGANIZATION  OF  THE  THESIS 

The  thesis  is  organized  as  follows.  In  Chapter  II  we  consider  the  problem  of 
fusion  in  DD.  Optimum  detection  with  quantized  sensor  data  is  considered  in  Chapter 
III,  where  the  Quantized  Detection  algorithm,  QD,  is  introduced.  Numerical  examples 
to  illustrate  the  algorithm  are  given  in  Chapter  IV.  Generalization  to  the  case  of  vector 
observations  is  presented  in  Chapter  V.  Optimum  regeneration  of  sensor  observations 
from  their  quantized  versions  and  another  sensor  observation  is  considered  in  Chapter 
VI.  A  summary  of  the  thesis,  conclusions  and  suggestions  for  future  research  are  given 
in  Chapter  VII.  Proofs  to  some  equations  and  FORTRAN  programs  to  calculate 
parameters  of  the  minimum  risk  and  the  minimum  distortion  quantizers  are  given  in 
the  appendices. 


II.  OPTIMUM  FUSION  OF  LOCAL  DECISIONS 


In  this  chapter  the  important  question  of  the  optimum  fusion  rule  will  be 
answered.  The  relationship  of  the  optimum  fusion  policy  to  the  ratio  of  costs  and  the 
correlation  coefficient  between  observations  is  determined. 

A.  INTRODUCTION 

Distributed  Detection  with  fusion  is  a  two  level  optimization  problem.  The 
problem  can  be  formulated  in  the  following  three  ways: 

1.  Local  Decision  Optimization 

The  first  way  is  to  select  the  fusion  rule  apriori  and  optimize  the  local  decision 
rules  accordingly.  Setting  the  activity  of  the  fusion  center  as  AND  fusion,  Tenney  and 
Sandell  [5]  derived  optimum  local  decision  rules  for  a  pair  of  spatially  separated 
detectors  with  conditionally  independent  observations.  They  prove  that  local  decision 
rules  are  simple  likelihood  ratio  tests  with  constant  thresholds.  The  thresholds  are  the 
solution  of  a  pair  of  coupled  algebraic  equations  that  correspond  to  the  global 
minimum  of  the  detection  cost  function.  They  also  show  that  for  the  case  of  correlated 
observations  local  decision  rules  are  likelihood  ratio  tests  but  with  data  dependent 
thresholds.  Functional  solution  of  the  threshold  equations  in  the  later  case  violates  the 
principle  of  decentralization.  Realizing  the  difficulty  of  the  problem  in  the  case  of 
correlated  observations,  Lauer  and  Sandell  [4]  designed  suboptimal  local  decisions  for 
AND  fusion.  Their  local  decision  rules  are  likelihood  ratio  tests  with  constant 
thresholds  satisfying  the  necessary  conditions  of  optimality.  Kovatana  [15]  considered 
AND  fusion  for  two  detectors.  Fef]ar  [16]  compared  AND  to  OR  fusion  for  two 
detectors.  He  claimed  that  OR  is  better  than  AND.  Stearns  [17]  contradicts  Fefjar's 
results.  He  showed  by  an  example  that  OR  combining  is  better  for  higher  cost  of 
missing  the  target  while  AND  combining  is  better  for  higher  cost  of  false  alarms. 

2.  Fusion  Rule  Optimization 

In  the  second  formulation  of  the  problem,  local  decision  rules  are  set  apriori. 
Optimization  is  carried  out  with  respect  to  the  fusion  rules.  An  example  of  this 
situation  could  be  factory  built  sensors  that  cannot  be  adjusted.  Assuming  local 
threshold  settings  Chair  and  Varshney  [8]  prove  that  for  the  case  of  conditionally 
independent  sensor  observations,  the  optimum  fusion  rule  is  a  likelihood  ratio  test  that 
sums  local  decisions  weighted  according  to  their  reliability. 


3.  Global  optimization  of  the  Local  decisions  and  the  Fusion  Rule 

The  third  formulation  involves  optimization  at  both  levels.  Here  local 
decisions  are  optimized  for  every  possible  fusion  rule.  The  optimum  fusion  rule  is  the 
one  that  minimizes  cost. 

The  main  issue  of  this  chapter  is  the  global  optimization  of  the  DD  system  for 
general  correlated  observations.  First  we  will  state  the  main  results  for  the  case  of  N 
conditionally  independent  and  identically  distributed  sensor  observations.  Then,  the 
problem  of  fusing  two  local  decisions  of  sensors  with  correlated  observations  is 
considered. 

B.  GLOBAL  OPTIMIZATION  OF  DISTRIBUTED  DETECTION 

In  CD  all  sensor  observations  are  available  at  one  central  processor  for  detection. 
The  decision  rule  in  CD  is  a  likelihood  ratio  test  in  the  observations  yj,y-,,...,y^..  It 
declares  H,  is  true  if  the  likelihood  ratio 

A  (y1(y2,...,yN  )  >  C  ,  (2.1) 

otherwise  it  will  declare  HQ  to  be  true. 

In  DD  only  local  decisions  are  sent  to  the  central  processor  (  fusion  center).  The 
objective  of  the  fusion  center  is  to  mix  (  fuse  )  the  local  decisions  into  a  single  global 
decision  with  minimum  decision  cost.  So  given  the  local  decisions  the  observation 
space  of  the  fusion  center  consists  of  2^  discrete  points.  The  activity  of  the  fusion 
center  is  to  divide  this  space  into  two  decision  regions  ZQ  and  Zj.  The  decision  rule  of 
the  fusion  center  is  a  likelihood  ratio  test  [S.]  The  fusion  center  declares  Hj  is  true  if 

A  (u1,u2,...,uN  )  >  C.  (2.2) 

otherwise  it  will  declare  that  HQ  is  true.  In  the  special  case  of  conditionally 
independent  and  identically  distributed  observations,  the  fusion  rule  is  a  k  out  of  X 
rule.  Reibman  and  Nolte  [9]  considered  this  problem.  Assuming  the  same  decision  rule 
for  every  detector  they  optimize  local  decisions  for  every  k,  k=  1,2,. ...N  then  pick  the  k 
with  the  minimum  decision  cost. 

If  sensor  observations  are  not  conditionally  independent,  there  is  no  guarantee 
that  local  decisions  arc  simple  likelihood  ratio  tests.  The  problem  turns  out  to  be  XP- 


complete  which  needs  exhaustive  enumerations  to  find  the  optimum  decision  rules  [14  ] 
Moreover  if  sensor  observations  are  not  identically  distributed,  there  are  as  many  as 
221  possible  fusion  rules  for  the  N  sensor  decisions.  Any  algorithm  that  goes  through 
the  entire  fusion  list  optimizing  local  decisions  will  be  impractical3  for  N  £  6. 


Our  approach  to  avoid  this  exhaustive  enumeration  is  the  following: 

1.  We  assume  that  local  decisions  are  likelihood  ratio  tests  with  constant 
thresholds.  Again  we  emphasize  that  this  assumption  is  valid  only  for 
conditionally  independent  observations,  there  is  no  guarantee  that  it  is  correct 
for  correlated  observations  [5].  So  the  constant  threshold  likelihood  ratio  test  is 
optimum  for  conditionally  independent  observations  and  perhaps  suboptimum 
for  correlated  observations.  However  the  solution  tends  to  the  optimum 
solution  as  the  correlation  coefficient  tends  to  zero  [4]. 

2.  Those  fusion  rules  which  agree  with  the  CD  solution  will  be  tested.  The  rest  of 
the  fusion  rules  will  be  disregarded.  The  meaning  of  this  will  be  made  clear  in 
the  following  example. 


Let  us  consider  the  case  of  two  sensors  (N  =2  )  in  detail.  To  be  explicit, 
consider  detection  of  known  signals  in  gaussian  noise.  The  sensor  observations  are 
given  by: 


H0:  Vj  =  n;  ,  i- 1,2  (2.3) 

and 

Hj  :  yj-ai  +  n.,  i-  1,2.  (2.4) 

The  a;'s  are  positive  constants  and  N  =  [n1  n2]c  is  vector  of  zero  mean  with 
covariance 


K  ’  [p  3  <2'5> 

where  p  is  given  by 

p  =  E  {  n,  n2  }.  (2.6) 


3 A  computer  that  spends  1  ft  second  in  every  optimization  process,  will  spend 
40000  years  to  determine  the  optimum  fusion  rule,  for  N  =  6. 
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The  threshold  equation  of  the  CD  problem  is  given  by  [6], 

(ax  -p  a2  )yj  +  (a2  -  p  a {  )y2  -(at2  +  a22  -2p  a,  a2)/2  +  (l-p2  )log(C)  (2.7) 

which  is  a  straight  line  in  the  Vj  y2  plane.  Figure  2. 1  shows  decision  rules  based  only 
on  Dj,  only  on  D2,  both  decision  rules  together,  and  the  decision  rule  of  CD. 

The  global  optimization  requires  optimizing  local  decision  rules  for  even'  fusion 
rule  then  picking  the  fusion  rule  with  minimum  average  cost.  The  observation  space  of 
the  fusion  center  consists  of  four  discrete  points  (0,0),  (0,1),  (1,0),  (1,1).  Any  fusion 
rule  divides  this  space  into  two  decision  regions  Zj  and  ZQ.  There  are  24  =16  methods 
to  subdivide  four  points  into  two  groups.  Table  1  contains  a  list  of  those  fusion  rules. 
Some  special  cases  for  the  detection  problem  are  as  follows. 

1.  If  Cl0  -*  co,  i.e.  the  cost  of  missing  the  target  is  extremely  high,  the  CD 
solution  assigns  all  the  observation  space  to  ZQ.  The  fusion  center  can  perform 
the  same.  This  is  fusion  rule  one. 

2.  Similarly  if  CQ1  -»  °o,  the  fusion  center  will  always  decide  H.,  this  is  fusion  rule 
two. 

3.  If  a2  =  p  at,  the  CD  will  decide  based  only  on  y..  So  will  the  fusion  center. 
This  is  fusion  rule  three.  This  can  only  happen  when  a.{  ^  a2. 

4.  If  ax  =  p  a,,  the  CD  will  decide  based  on  y,.  This  is  fusion  rule  four.  This  can 
only  happen  when  a,  £  ar 

The  first  two  situations  represent  extreme  conditions  of  C.  The  next  two  conditions 
deal  with  specific  values  of  p.  We  also  distinguish  the  following  two  cases. 

Case  a 

-1  ^  p  ^  min(  at  ,a2  ),'max(  at  ,a2  ). 

In  this  case  the  Vj  and  y2  intersections  of  the  threshold  equation  (2.7  )  are  of  the  same 
sign. 

Case  b 

min(  at  ,a2  )/max(  at  ,a2 )  <  p  <  1. 

In  this  case  the  yt  and  y2  intersections  of  the  threshold  equation  are  of  different  signs. 
We  shall  consider  these  intervals  of  p  when  we  study  the  effect  of  correlation  between 
sensor  observations. 

The  CD  threshold  in  the  yt  y2  plane  suggests  assigning  the  decision  point  (0.0)  to 
Zg  and  (1,1)  to  Zy  The  fusion  rules  from  5  to  14  do  not  do  this.  They  either  assign 
(0,0)  to  Zj  or  assign  (1,1)  to  ZQ  or  assign  (0,0)  and  (1,1)  to  the  same  decision  region. 


TABLE  1 

EXHAUSTIVE  FUSION  LIST  OF  TWO  DECISIONS 


Rule  s 

Z* 

Z, 

Comments 

1 

<t> 

(0.0).(0,l).(l.0).(U) 

C  — #  00 

2 

(0.0).(1.0),(0,l).(l.l) 

a 

3 

(O.O).(l.O) 

(o.n.(i.i) 

4 

(O.O).(O.i) 

(i.o).(t.n 

“Ct 

5 

(o.n 

(O.ot.d  o).(i.D 

6 

(i.»» 

(O.O).(O.l ).( I.l ) 

i 

i 

i 


7 

d.i) 

(0.*)).(  I.O).(O.l) 

1  1 

$  |  (O.m  1.0)  I  (D.m.d.n 


9  I  (D.iui.n  |  ('>",.1,1.11) 


10 

(i.O).(i.n 

1 

io.m.111,1) 

1 

11 

<0.01.1  1.1 1  j 

1 

(O.d.d.o) 

i 

i: 

(O.lMl.'M.ll.ll  | 

(0.0) 

i 

13 

1  (O.'OJO.I).'  I.l)  j 

(1.0) 

j 

U 

1  (O.'ii.,  I."!.d.l ) 

i  i 

(‘1  1 ) 

'' 

15 

i  1 

!  (O.O).im.i  [.in 

d.l) 

AND 

; 

We  will  not  consider  these  ten  fusion  rules  further.  The  remaining  two  decision  rules 
are  the  AND  fusion  and  the  OR  fusion.  Let  us  now  consider  optimizing  each  of 
them. 

1.  '  AND  '  Fusion 

In  AND  fusion  uo  is  given  by: 

u0  =  Uj  u2.  (2.8) 

The  individual  rules  are  given  by  assigning  to  Zj  if 

y,  £  T;  ,i  =  1.2.  (2.9) 

Otherwise  thev  assign  it  to  Zn. 

The  probability  of  detection  Pd(AND)  and  probability  of  false  alarm  P^AND) 
of  the  fusion  center  are  given  by: 

Pd  (AND)=  L  tfVj.y,  /Hj  )  dVj  dy2  (2.10) 

ll  l2 

and 

Pf  (AND)- L°  J°°  «ylty2  ,'Hg  )  dy-j  dv,  (2.11) 

1 1  1  2 

It  has  been  shown  in  Chapter  I  that,  to  withm  positive  multiplicative  and 
additive  constants,  the  average  decision  cost  is  given  by 

R  =  1  +  C  Pf-  Pd.  (2.12) 

Substituting  for  Pd  and  Pf  in  (2.12)  from  (2.10)  and  (2.11)  expresses  R(AND)  as  a 
function  of  Tj  and  T2.  The  necessary  conditions  for  optimality  are 

d  R .5  Tj  =  0  and  d  R  d  T2  =  0  ,  (2.13) 


which  can  be  written  in  the  forms: 


(2.14) 


QU  UU 

c  JT  flT,,y2  ,'H0)  dy2  =  ^  HT,,y2  / Hj  )  4y, 


cj  fly,,T,  ,'H0)  dy,  -  j  Ry,.T2  /H|  )  dy, . 

I  I  A  I 


(2.15) 


Applying  Bayes  rule  and  rearranging  terms,  one  can  write  (2.14)  and  (2.15)  as  follows: 


JT  fl>yTi  .H0)  dy, 

W  =  C“^ - 

J  f(y2.  Tj  ,Hj)  dy; 
» 


(2.16) 


A,  (T,)«  C 


iT  f(}'i  T,  .Hq)  dy i 

_ M _ 

Jt  ^>1  T2  ’Hl>  d>'l 

l  1 


To  insure  minima  the  Hessian  matrix  of  R  with  respect  to  Tj  and  T, 


(2.17) 


p2R.dTj2 

H  L52R.5T23T1  52R.dT22J 


(2.18) 


(2.19) 


must  be  positive  definite.  Optimum  threshold  settings  T;  and  T,  are  the  solution  of 
(2.16)  and  (2.17)  that  corresponds  to  the  global  minima,  so  all  possible  solutions  of 
(2.16)  and  (2.17)  must  be  tried.  The  coupling  between  (2.16)  and  (2  17)  to  determine 
the  thresholds  represents  the  cooperation  that  can  occur  between  the  two  local 
detectors  to  minimize  the  overall  decision  cost. 
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2.  'OR'  Fusion 

The  decision  of  the  OR  fusion  is  given  by 


uo  =  ui  +  ^  -  ui  u2- 


(2.20) 


The  probability  of  detection  Pd  (OR)  and  probability  of  false  alarm  Pf  (OR)  are  given 
bv 


Pd  (OR). 


=  1  ’  /Hi  >  dyi  dy: 


(2.21) 


Pf(0R)=  1-  jT^  flyl(y2  ;H0  )  dy,  dv. 


(2.22) 


while  the  necessary  conditions  for  optimality  are 


A  (T,  )  =  C 


&  H»  T,  .H„)  dy, 
fj2  ffyj.T,  .H,)dy2 


(2.23) 


A  IT,)-  C 


U  Ity./Tj  .H„)  dy, 
ll1  «y,.'T2.H1)dy1 


Again  the  Hessian  matrix  must  be  positive  definite. 

3.  Solution  of  the  Nonlinear  Tlireshoid  Equations 

The  pair  of  coupled  equations  (2.16),  (2.17)  for  the  AND  fusion  and  (2.23) 
and  (2.24)  for  the  OR  fusion  can  be  solved  using  Max's  technique  [18].  The  technique 


is  summarized  as  follow:  pick  a  value  of  Tj  and  calculate  T2  from  (2.16)  or  (2.23  ).  If 
the  calculated  value  of  T2  does  not  agree  with  that  value  calculated  from  (2.17)  or 
(2.24)  then  Tj  must  be  chosen  again.  This  approach  is  time  consuming.  Another 
approach  is  the  method  of  successive  substitution  [19).  We  first  put  the  two  equations 
in  the  form 


T,  =  G(T,  fT,  )  ,  T. 
1  k  +  l  !k  2k  2 


-  F(T.  ,T,  ) 
k+ 1  *k+l  ‘k 


(2.25) 


then  start  with  a  reasonable  guess  for  (T^  )0  and  (T2  )Q.  A  suitable  initial  guess  is  the 
locally  optimum  solutions,  i.e.  the  thresholds  that  would  optimize  the  detection  if  each 
sensor  works  alone.  These  will  be  denoted  by  Tllo  and  T2]q.  For  known  signals  in 
gaussian  noise  these  are 


(Ti  )0  =ai  2  +  log(C)  a.  . 


(2.26) 


4.  Numerical  Results 

We  have  solved  the  threshold  equations  for  both  fusion  rules  for  a}  =1.7  and 
a-,=  2.3  for  several  values  of  p  and  C. 

To  compare  AND  and  OR  fusion,  define  K  as  the  ratio  of  the  AND  cost  to 
the  OR  cost. 

R  _  1  -  CP,  (AND)- P,  (AND) 

1  t  CPf  (OR)  -  Pd  (OR) 

We  have  also  computed  the  Receiver  Operating  Characteristic4  (ROC)  curves  of 
classical  communication  theory  (20)  for  each  fusion  rule. 

Figure  2.2  shows  the  ratio  K  as  a  function  of  C  for  p  =  0,  0.2  ,  0.4.  The 
figure  shows  that  AND  fusion  is  optimum  for  C  2:  1  and  OR  fusion  is  optimum  for 
lower  values  of  C.  The  same  is  clear  from  Figure  2.3;  ROC  curves  of  AND  fusion  are 
above  those  of  OR  fusion  for  C  £  1  and  lower  otherwise.  The  performance  difference 
becomes  smaller  as  the  correlation  coefficient  increases.  Also  the  figures  show  that  the 
performance  degrades  for  both  fusion  rules  as  p  tends  to  one.  This  is  in  sharp  contrast 
to  CD  which  has  perfect  detection  for  p=  I. 

4Pd  as  a  function  of  Pf 


The  same  effects  can  be  concluded  from  Figure  2.4  and  Figure  2.5.  Figure  2.4 
shows  the  ratio  K  as  a  function  of  C  for  aj  =  1  and  a2  =2  and  for  p  =  0,  0.25,  0.5. 
Figure  2.5  shows  the  ROC  curves  for  both  fusion  rules  for  the  same  case.  The  figure 
shows  that  AND  fusion  is  optimum  for  C  £  1  and  OR  fusion  is  optimum  for  lower 
values  of  C. 

C.  THE  EFFECT  OF  CORRELATION  BETWEEN  SENSOR  OBSERVATIONS 

So  far  we  have  answered  the  question  of  the  optimum  fusion. rule.  For  C  £  1 
AND  fusion  is  optimum.  Let  us  now  examine  the  effect  of  the  correlation  coefficient  p 
on  the  performance  of  AND  fusion  for  C  ^  1  (its  range  of  superiority).  We  assume 
without  loss  of  generality  that  a2  is  greater  than  ar  The  two  necessary  conditions  for 
optimality  of  AND  fusion  are  (2.16)  and  (2.17).  For  the  problem  of  known  signal  in 
gaussian  noise  these  can  be  written  as: 


At  (Tt  )  = 


:/-Vf 

.  f  T,  -a, 

l  V( 


h±L_\ 

v  d-p:)  J 

2  ~a2  ~P  (T~i  -a1  ) 


(2.2S) 


d-p2) 


A2  (T2  )  = 


r  t,  -P  t2  \ 

tvd-P2)  ) 

(  Tt  -a,  -p  (T:  -a,  ) 
1  V  (i-p2) 


(2.29) 


Notice  that  C  appears  only  as  a  multiplicative  constant  in  the  two  equations.  The  role 
of  p  is  not  that  obvious.  Examining  the  two  equations  leads  to  the  following  insights 
about  the  role  of  p: 

1.  Tj  =-20  and  T2  =T21o  is  a  solution.  This  corresponds  to  the  decision  rule  of 

d2. 

2.  T,  =  -2C  and  Tt  =  Tllo  is  a  solution.  This  corresponds  to  the  decision  rule  of 


3.  If  a2  is  greater  than  a}  we  expect  the  performance  of  D,  alone  to  be  better  than 
that  of  Dj  alone  and  that  of  the  selfish  decision  rule  in  which  each  detector  tries 
to  minimize  its  own  detection  cost,  not  the  system  decision  cost,  by  using  T,.  . 


We  now  prove  three  lemmas  concerning  these  equations. 

1.  Lemma  1. 


For  p  ^  a1  / a2. 


T  <  T 
ll  “  lllo 


T  <  T 

l2  ~  1 21o 


where  TUo  is  the  optimum  threshold  of  the  ith  detector  operating  alone. 

Proof: 

Since  the  argument  of  the  complement  of  the  error  function  in  each 
denominator  is  less  than  the  argument  in  the  corresponding  numerator,  the 
fraction  is  always  less  than  one.  This  implies  that 

A;  (T)  £  C,i=l,2. 

2.  Lemma  2 

For  p  ,a2,  the  only  possible  solution  of(2.2S)  and  (2.29)  is: 

T  =  T 

1  2  1  21o 


Tj  =-oo. 


Proof: 


For  p  =al  ,'a2  equation  (2.29)  becomes 


A,(T,)  =  C  =  A2  (T2;o).  (2.30) 

The  corresponding  value  ofTj  is  Tj  =-oo. 

3.  Lemma  3 

For  p  >  aj  /a,  the  optimum  solution  for  Tj  and  T2  is: 

Ti  -30 

and 

T2  "  Tao. 

This  means  that  the  decision  of  the  optimum  AND  fusion  is  that  of  D,. 

Proof: 

Recall  that  the  CD  threshold  line  divides  the  observation  space  into  two 
decision  regions.  For  positive  signals  the  following  inequality  is  satisfied  in  the 
reaion  to  the  naht  of  the  CD  line: 


C  >>'2  /H0  tf>'i  A2  /H,  ) 


(2.31) 


The  converse  of  this  inequality  is  true  in  the  left  region.  The  decision  region  Z1 
of  any  other  decision  rule  contains  areas  from  the  right  and  from  the  left  of  the 
CD  line.  Areas  to  the  right  will  have  a  negative  contribution  to  the  decision 
cost  while  areas  to  the  left  will  have  positive  contributions.  Now  assume  that  Tj 
and  T2  ,  where  both  are  finite,  satisfy  the  necessary  condition  (2.28)  and  (2.29). 
We  shall  prove  that  they  cannot  correspond  to  the  optimum  solution.  The 
finite  point  (  Tj  ,T2  )  either  lies  to  the  left  or  to  the  right  of  the  CD  threshold 
line  as  shown  in  Figure  2.6  a  and  b  respectively.  In  Figure  2.6  a  the  intersection 
of  the  CD  line  with  the  line  Vj  =  Tj  is  a  better  solution  since  it  excludes  an 
area  in  which  C  ffyt  ,y,  ;HQ  )  is  greater  than  ffvj  ,  y2  /Hl  ).  A  better  solution 
than  this  has  the  same  T2  but  with  Tj  =  -so  since  the  added  area  has  negative 
contribution  to  the  cost.  In  Figure  2.6  b,  Tj  =-oo  and  T'2  is  a  better  solution 
than  Tj  and  T.,,  since  the  added  area  has  a  negative  contribution  to  the  cost. 
In  both  cases  Tj  =  -so  is  the  optimum  solution  and  the  corresponding  optimum 
value  of  T2  is  T,(o. 

As  a  result  of  the  above  three  lemmas  it  is  clear  that 


1. 


2. 


Anv  solution  of  the  necessarv  conditions  must  satisfv 


Tl  * 

T,  ^ 


Tlio  and 

T2,o 


The  performance  of  the  AND  fusion  saturates  to  that  of  D-,  alone  for  p  ^ 
aj/a,.  We  might  recall  that  the  threshold  line  of  the  CD  system  changes  slope 
at  that  value  of  p.  We  will  refer  to  this  value  of  p  by  pcr.  This  result  is  in 
contradiction  with  Lauer  and  Sandell's  results  [4]  which  shows  performance 
continuing  to  degrade  with  increasing  p  for 


p  >  p 


cr 


Limiting  behavior  for 


P-1- 


For  p  =-l  the  joint  probability  density  function  flNj  ,y,  'H0  )  has  values  only  on  the 
line  Vj  =  -yv  So  any  threshold  values  Tj  and  T,  such  that  Tj  =  -T,  will  produce 
AND  fusion  with  zero  probability  of  false  alarm.  This  can  be  visualized  from  Figure 


2.7  .  Consequently,  Pj  will  be  given  by 


P,  =  0.5  erfc 


|t2  -a2  | -0.5  erfc  |  T2  +  at  j 


Maximizing  Pu  with  respect  to  T2  yields 


T2  =(a2  -aj)/2  . 


(2.33) 


For  the  special  case  of  equal  SNR  sensors.  T,  =  0. 
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D.  NUMERICAL  RESULTS 


The  average  decision  costs  vs  p  for  aj  =  a2  =  2,  and  C  =  1  are  shown  in  Figure  2.8. 
Threshold  values  Tj  and  T,  vs  p  for  the  same  case  are  shown  in  Figure  2.9.  Figures 
2. 10  and  2. 1 1  show  the  same  for  C  =  10. 

These  four  figures  for  the  case  of  equal  signal-to-noise  ratio  show  that  the  two 
detectors  cooperate  with  each  other  using  the  same  decision  rule  (  equal  thresholds  ). 
Their  threshold  is  an  increasing  function  of  p.  The  limit  of  this  threshold  as  p  — ►  *1  is 
zero.  This  behavior  agrees  with  (2.33).  The  limit  of  the  threshold  as  p  -*  1  is  Tj  .  This 
is  because  for  p  -»  1  the  two  systems  have  identical  observations. 

The  detection  cost  curves  show  that  the  cost  is  an  increasing  function  of  p.  The 
curve  of  the  AND  fusion  has  the  same  shape  as  the  curve  of  the  CD  system.  Both 
systems  attain  their  best  performance  at  p  =  -l.  They  have  the  same  worst  performance 
for  p=  l 

Figures  2.12  and  2.13  represent  the  case  of  unequal  SNR  sensors  for  C=l. 
Figures  2.14  and  2. 15  show  the  same  for  C  =  10. 


0.5 


0.5 


These  four  figures  for  the  case  of  unbalanced  sensors  show  that,  the  two 

detectors  cooperate  using  different  thresholds.  The  threshold  of  the  higher  signal-to- 

noise  detector  is  an  increasing  function  of  p  while  the  other  threshold  is  a  decreasing 
function  of  p. 

The  cost  curves  show  that  the  fusion  rule  has  its  best  performance  at  p  =-l. 
Both  DD  and  CD  have  their  worst  performance  at  p=  pCT.  For  p  >  p^  the 
performance  of  the  optimum  fusion  rule  is  the  same  as  the  detector  of  the  higher 
signal-to-noise  ratio.  Recall  that  CD  system  has  perfect  detection  for  p  =  1  when  the 
SNR's  are  unequal.  As  C  increases  the  average  cost  of  each  system  increases.  This  can 
be  explained  from  the  expression  for  R  in  which  the  probability  of  false  alarm  is 
weighted  by  C. 

E.  DISCUSSION  AND  CONCLUSIONS 

We  have  shown  that  the  optimum  fusion  rule  is  determined  by  the  ratio  of  costs 
and  the  apriori  probabilities.  For  equal  error  costs  AND  and  OR  fusion  rules  are 

equivalent.  This  is  not  surprising  since  each  system  turns  out  to  be  the  minimum 

probability  of  error  detector,  thresholds  are  adjusted  such  that  I  *Pd  =  Pf  It  might  also 
be  noted  that  the  optimality  of  the  fusion  rule  is  independent  of  the  correlation 
coefficient  and  the  signal-to-noise  ratio  in  this  case.  We  also  note  that  the  detection 
cost  of  the  optimum  fusion  rule  has  its  minimum  value  at  p  =  -l.  It  has  its  maximum 
value  at  p  =  aI,  a2.  The  performance  saturates  at  the  cost  of  decision  of  the  detector  of 
higher  SNR.  In  the  interval  (p  e  [a1,  a2, 1  j),  the  optimum  fusion  rule  ignores  the 
decision  of  the  detector  of  lower  SNR.  As  a  good  dynamical  example  that  agrees  with 
this  result  is  the  switched  diversity  combiner  [21]  in  fading  environments  and  its 
centralized  counterpart,  the  maximum  ratio  diversity  combiner  [22].  Recall  that  for 
unequal  SNRs  the  performance  of  the  CD  system  improves  in  this  interval  and  has 
perfect  detection  for  p  =  1.  Also  it  is  important  to  note  that  the  optimum  thresholds 
of  the  individual  observers  are  not  the  same  as  if  they  were  operating  independently, 
but  must  be  determined  by  simultaneous  solution  of  two  coupled  nonlinear  equations. 
This  represents  the  cooperation  between  the  two  detectors  to  work  as  a  team.  Lastly 
the  performance  difference  between  CD  and  DD  is  due  to  the  information  loss  in  local 
data  processing,  however  DD  has  fewer  requirements  on  the  communication  channel 
in  contrast  to  CD  which  requires  infinite  bandwidth.  A  compromise  between  these  two 
extremes  is  to  allow  more  information  than  just  decisions  to  be  sent  to  the  fusion 
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center.  This  is  the  concept  behind  the  Quantized  Detection  algorithm  considered  in  the 
following  two  chapters. 


«.  V 


III.  DETECTION  USING  QUANTIZED  SENSOR  OBSERVATIONS 

A.  INTRODUCTION 

So  far  detection  with  sensor  observations  has  been  described  using  two  methods. 
In  the  first  method  all  sensor  observations  are  sent  to  some  central  processor  which 
makes  a  decision  based  on  a  likelihood  ratio  test.  In  the  second  method  only  local 
decisions  are  sent  to  the  central  processor  which  fuses  these  decisions  into  a  global 
decision.  While  the  first  method  is  very  easy  to  design  it  requires  in  principle  infinite 
bandwidth  communication  channels.  The  second  method  requires  only  one  information 
bit  per  detection.  Detection  with  quantized  sensor  observations  will  be  introduced  in 
this  chapter.  The  main  goal  of  the  chapter  is  to  grade  the  road  from  the  DD  problem 
to  the  CD  problem.  It  will  be  referred  to  by  Quantized  Detection.  QD.  The 
performance  improvement  of  the  DD  problem  will  be  traced  as  the  amount  of 
information  delivered  to  the  fusion  center  increases. 

First  let  us  consider  the  problem  of  the  Primary  Decision  Maker  (PDM)  and  its 
quantized  second  opinion  (consultant).  We  will  prove  three  theorems  concerning  the 
decision  rule  of  the  PDM.  Then  fusion  of  two  quantized  observations  of  an  arbitrary 
number  of  levels  will  be  considered.  As  a  special  case,  fusion  of  two  sensor 
observations,  one  quantized  to  N  levels  and  the  other  to  N  +  1  levels,  will  be  proven 
equivalent  to  the  PDM  and  an  N-levei  quantizer.  Comparison  between  different 
configurations  will  follow. 

B.  TEAM  DECISION  OF  A  PRIMARY  DECISION  MAKER  AND  A  SECOND 

OPINION  QUANTIZER. 

1.  Formulation  of  the  PDM  Problem 

Consider  the  structure  of  Figure  3.1  in  which  >fj  is  quantized  into  Vj  by  the 
quantization  rule  a  of  N  levels. 

a  ’  Yj  Ylq  •  (3-D 


The  primary  decision  maker  will  make  his  decision  u  ,  about  the  phenomena  H  based 
on  its  own  observation  y,  and  the  quantized  observation  y. 


The  problem  of  the  PDM  is 

1.  to  design  the  quantization  rule  a  i.e.  to  specify  the  set  of  N  points 

-oo  =  Xt  <  X2  <  ...  <  XN  <  oo 
that  defines  the  quantizer  intervals,  and 

2.  to  design  the  decision  rule  y2 


h  ■■  YIo  X  Y2  -*  (  0  -  1  ) 


in  order  to  minimize  the  decision  cost. 

2.  Problem  Analysis 

Our  approach  is  as  follows.  We  first  design  the  optimum  Bayes  decision  rule 
given  a  set  of  quantizer  parameters.  Next,  the  average  cost  is  expressed  as  a  function  of 
these  parameters.  We  then  minimize  the  average  cost  with  respect  to  them. 
a.  The  Optimum  PDM  Given  Some  Quantization  Rule  a 

We  have  shown  in  Chapter  I  that,  to  within  an  additive  and  a 
multiplicative  positive  constant  the  average  cost  is  given  by  [6] 


R  =  CPf-Pd 


where  C  is  the  ratio  of  error  costs  and  Pf-  and  Pd  are  the  probability  of  false  alarm  and 
probability  of  detection  respectively.  The  PDM  receives  a  quantized  level  ylq  =  Q.  . 
He  will  make  his  decision  on  the  basis  of  his  own  observation  y2  and  ylq  .  The 
performance  of  the  the  primary  decision  maker,  given  some  quantization  rule  a,  is 
given  by  the  following  lemma. 

Lemma  3.1 

The  probability  of  detection  and  probability  of  false  alarm  of  the  Primary  Decision 
Maker  are  given  bv: 


ki  +  ‘lez5WH',dy',ly> 


iw-ir r 


It* 


m 


*\.V 

Vh/. 

& 

Ro? 

■yjS* 

i 


y': 


i 

>■. 

k£j 

,*v. 


v  •. 

#.» 

M 

s& 


Pf  =  £  J  v  +  1  L  =  7^yl’y2  /H0  )  dyi  dy2 


i-1  y2  e  zVi 


(3.5) 


where  Z,j  is  the  decision  region  Z,  given  that  y^X^X^  J. 

Proof: 

The  proof  is  given  in  Appendix  (A). 

The  decision  rule  of  the  Primary  Decision  Maker  is  given  by  Theorem  3.1. 
Theorem  3. 1 

Given  v,  and  y,  the  decision  rule  of  the  Primary  Decision  Maker  of  Figure  3.1  is 
1.  deterministic 


y2  •  vlq  x  y2  -*  ( o ,  i  ) 


2.  a  likelihood  ratio  test 


(3.6) 


1  if  A  (y2  )  >  0^  (y2  ) 
0  if  A  (y;  )  <  0.(y2) 


j-  1.2 . N 


(3.7) 


where  A  (v,  )  =  f(y2  H,  ),  ffy2,  H0  ) 

3.  the  threshold  function  0.  (v,  )  is  given  by 


0i(y2)=  C 


JxJ  +  1^yi/y2’Ho,dyi 


x. 


,j=  1.2....N. 


(3.S) 


rxj+1^yi;y2-Hi)dyi 


Proof 

We  first  insert  (3.4)  and  (3.5)  into  (3.3).  Each  term  of  the  detection  cost  (3.3)  is  then 
given  by 


X 


Rj  “  J  z  1  x,+  t  C  R>'i  -y2  H0).  fly,  ,y2  H,  )  ]  dy,  dy, 

'2  ^tj  j 


To  make  R.  in  (3.9  )  negative  an  optimum  decision  rule  assiens  v,  to  Z,  if 


(3.9) 
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c  y  «yi  .y2  'Ho  >  dyi  ’  x  nyi  j2  /H» }  dyi  -  0  J"  ,-2’~N 


(3.10) 


otherwise  it  will  assign  y2  to  ZQ  . 

Applying  Bayes  rule  and  rearranging  terms,  decision  rule  (3.10)  can  be 


written  as 


Jxi  +  ^yi,/y2-Ho)  dyl 


A(y2)  ^  C  - ,  j~  1,2, ...N 

Jvi  +  ^yl/y2-Hi)dyi 


(3.11) 


which  completes  the  proof. 


b.  Optimum  Quantization  of  Y  j 

According  to  Theorem  3.1,  the  decision  rule  of  the  PDM  is  a  likelihood 


ratio  test  with  data  dependent  threshold.  The  threshold  depends  on  the  choice  of  X/s. 


To  find  an  optimum  solution  for  the  Xj  's  is  not  any  easier  than  that  of  the  DD 
problem.  Recall  that  for  the  DD  problem  optimum  solutions  are  possible  only  for  the 
case  of  conditionally  independent  observations.  Only  suboptimal  solutions  are  possible 
for  the  case  of  correlated  observations.  We  will  not  expect  more  for  the  QD  problem. 
Let  us  consider  each  case  separately. 

3.  Conditionally  Independent  Observations 

Under  the  assumption  of  conditionally  independent  observations,  i.e. 


fbyy2.H  )  =  fly  h  ) 


(3.12) 


the  decision  rule  of  the  Primary  Decision  Maker  can  be  simplified.  This  decision  rule  is 
given  bv  the  following  corollarv  of  Theorem  3.1. 


Corollary  1 


Assuming  conditionally  independent  sensor  observations,  and  given  yt  and  y2,  the 


decision  rule  of  the  Primary  Decision  Maker  of  Figure  3.1  is 


1.  deterministic 


Y:  =  Yiq  X  y2  -  (  °  ,  1  ) 


(3.13) 


mzmm 


2.  a  likelihood  ratio  test 


u 


if  A  (y2  )  £  ©j 
if  A  (y, )  <  0j 


,j=  I,2,...,N 


where  A  (y2  )=  Ry2  /Hj  )/f(y2/H0  ) 
3.  the  threshold  0.  is  given  by 


jJHy./HoXiy, 

Ai _ 

J^i  +  lHy1;H,)dy1 

Aj 


,j  =  1,2,. ..N  . 


(3.14) 


(3.15) 


Proof 

By  applying  condition  (3.12)  in  the  threshold  equation  (3.8  )  one  obtains  (3.15)  which 
completes  the  proof. 

Let  us  denote  the  conditional  probability  of  detection  and  the  conditional 
probability  of  false  alarm  of  the  PDM  given  that  the  quantization  level  of  y(  is 
received  by  Pd.  and  P^.  Let  T.  be  the  set  of  all  points  y2  for  which 

A2(y2)>©.  (3.16) 

Then  P,.  and  Pr  can  be  written  as 
dj  fj 

p0i  -  J  ' H1 )  dy2  <3-'7> 

j 

and 

pr,  'J™  ^2H0)dy2  (3.18, 

J 

Equations  (3.4)  and  (3.5)  are  now  given  by 


(3.19) 


pd  li;  fx!^yi/Hi)dyi  J  Ky2  /Hj  )  dv2 
and 

pf -■£  LX‘*i!y, /H0 ) ^y, f/y2 /h„ ) d>-2 ■  (J-») 

*  j 

Substituting  (3.19)  and  (3.20)  in  (3.3),  then  differentiating  R  with  respect  to  X^ 
,{j  =  2,3,...N}  will  yield  necessary  conditions  for  optimality  of  the  set  of  N  equations. 


c  OXk  ,y,  ,  H„  )dy,  -f  «xk  ,y,  ,'H„  )dy,  J- 
r  k  *  k-1 


[  |JIXk,y2  Hi  )dy2-f  f(Xk,y2  H1)dy2]  =  0  ,k=2,3 . : 

*k  rk  1 


Applying  Bayes  rule  and  rearranging  terms,  (3.21  )  can  be  written  in  the  following  way. 


A(Xk)  =  C 


fI>VHo)dy2-JTfIy2,H0)dy2 

JZ  dy2  -  J  i(y2  Hl)dy2 

1  k  rk-l 


,k=  2,3 . N 


(3.22) 


The  set  of  N-l  necessary  conditions  (3.22)  are  general  for  any  statistics  of  Vj  .  For  the 
special  case  when  A  (y,  )  is  mono  tonic  in  y,  ,  let  Tj  be  the  value  of  y2  for  which 


0j  =A;(Tj)  ,j  =  1,2 . N. 


(3.23) 


So  T.  is  given  bv 

j 


J^y,  H0)dy, 

Adp-C— j-1 -  ,j=  I.2....N  (3.24) 

J'i^y^I^dy, 


For  this  case  of  monotonic  A  (y,  )  the  set  of  necessary  conditions  for  optimality  (3.22) 
can  be  written  as 

CO  00 

fT  d?2  •  I  t^VHo)  d>2 

A(Xk)=C"^ - ^ -  .k-2,3 . N  (3.25) 

J  ffy^Hj)  dv2  -  f  flyyHj)  dy2 
lk  1  k-1 

Equivalently  we  can  write  (3.25)  in  the  form 


dy2 

A(Xk)  =  C-J* - 

J^'^jyHj)  dy2 


,k.=  2,3,... ,N 


Pd  and  ?.  in  this  case  are  given  by 


(3.26) 


H^dy,  J  Ky,  H,  )dy. 


and 


(3.27) 


pr  =  |,  j£‘+l  «y,  H0  )  dy,  fV,  :H0 )  dy. 


(3-28) 


Equations  (3.24)  and  (3.26)  are  only  necessary  conditions  for  optimality  for 
monotonic  likelihood  ratio.  They  correspond  to  minima  if  the  Hessian  matrix 
[.d"R.  dXjdXj]  is  positive  definite.  All  solutions  must  be  checked  for  the  global  minima. 

4.  Solution  of  the  Primary  Decision  Maker  Problem  with  Independent  Sensor 
Observations  and  Monotonic  Likelihood  Ratio 

The  following  theorem  summarizes  the  above  solution  of  the  PDM  with 
independent  sensor  observations  and  monotonic  likelihood  ratio. 
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Theorem  3.2 

The  decision  rule  of  the  Primary  Decision  Maker  with  a  Quantized  Consultant  (for 
independent  sensor  observations  and  monotonic  likelihood  ratio)  is; 

1.  deterministic 


Y2  :  Ylq  X  Y2  -  (  0  -  1  > 


(3.29) 


2.  a  likelihood  ratio  test 


u 


0 


f  1  if  A  (y,  )  >  0. 

1  ' 2  *  ,j=  1,2 . N 

\  0  if  A  (y,  )  <  0 


where  A  (v,  )=  fly,  IIj  );Ry2.  HQ  ) 

3.  the  threshold  function  0  (y,  )  is  given  by 


(3.30) 


dy, 

0j  -  C  - j-i -  j-l.:,...N.  (3.3n 

J 

The  optimum  set  of  quantizer  interval  end  points  must  satisfy  the  set  (3.26  ),  where 
Tk's  are  given  by  (3.24).  All  possible  solutions  must  be  checked  for  the  global 
minimum  cost. 

5.  The  Case  of  Correlated  Observations 

We  now  move  to  a  more  realistic  situation  by  removing  the  condition  of 
independent  sensor  observations.  In  many  radar  and  sonar  problems  noise  in  nearby- 
sensors  is  likely  to  be  correlated.  As  we  mentioned  before  the  decision  rules  (3.11)  are 
likelihood  ratio  tests  with  data  dependent  thresholds.  It  is  impossible  to  come  with 
their  optimum  functional  expressions  [4.]  A  suboptimal  solution  for  the  case  of 
correlated  observations  is  to  use  likelihood  ratio  tests  with  constant  thresholds  as  local 
decision  rules.  These  constant  thresholds  for  v,  are  the  values  of  y\  for  which  the 
inequality  (3.11  )  is  an  equality,  i.e.; 


65 


,k=  1,2 . N  • 


(3.32) 


Jx  foVVV  dYl 

A(Tk)=  C  - - 

j;k  +  1ayi/Tk.Hi)dyi 
Ak 

In  tenns  of  these  thresholds  Tk  's  and  the  quantizer  points  Xk  's  one  can  write 
expressions  for  the  probability  of  detection  and  the  probability  of  false  alarm  in  the 
form  of  (3.4)  and  (3.5).  Substituting  for  Pd  and  Pf  in  (3.3)  and  differentiating  R  with 
respect  to  Xk  for  k=2,3,...N  yields  the  following  set  of  necessary  conditions  for  the 
case  of  monotonic  A,  (v,  ) : 

J^k'1R>v'xIc.Ho)  d>: 

A  (Xk  )  =  C  -  ,k=  2,3 . N.  (3.33) 

J>%vxk.Hi)dy2 

The  set  of  equations  in  (3.32)  and  (3.33)  constitute  2N-1  equations  that  specify  the 
quantizer  interval  end  points  (Xfc  }  for  Vj  and  the  thresholds  (Tk)  for  v,  . 

C.  TEAM  DECISION  OF  TWO  QUANTIZERS  AND  A  FUSION  CENTER 

In  this  section  we  will  consider  the  problem  of  making  a  global  decision  based  on 
two  quantized  observations. 

1.  Formulation  of  the  QD  problem 

For  the  structure  of  Figure  3.2,  Vj  is  quantized  into  N  levels  by  the 
quantization  rule 

<7  -V  — ♦  V  (3.34) 

U1  •  1  1  1 lq  ' 

and  y,  is  quantized  into  M  levels  by  the  quantization  rule  a; 

:  Y,  -♦  Y2q  .  (3  35) 

The  quantized  values  and  v,q  are  sent  to  the  fusion  center  which  must  decide  which 
state  of  the  phenomena  is  true.  It  is  required  to  design  the  quantization  rules  Uj  and 
a,  and  the  decision  rule  y 


?:YlqXY2q-(°.l)  (3-36) 

to  minimize  the  global  cost. 

2.  Problem  Analysis  and  the  QD  Algorithm 

The  observation  space  of  the  fusion  center  contains  NM  points  to  be  divided 
into  two  decision  regions.  Since  there  are  as  many  as  fusion  methods,  checking  all 
of  them  will  consume  a  very  long  time  even  for  small  values  of  X  and  M.  A  suboptimal 
solution  is  to  approximate  the  threshold  equation  of  the  corresponding  CD  problem  by 
a  piecewise  curve  in  the  Vj  y,  plane.  This  is  illustrated  in  Figure  3.3. 

The  figure  shows  a  schematic  diagram  of  a  CD  threshold  curve  and  its 
staircase  approximation.  The  approximate  curve  consists  of  segments  of  straight  lines 
connected  together.  The  coordinates  of  the  connecting  points  will  play  the  role  of  the 
interval  end  points  of  the  quantizers.  Let  us  first  write  an  expression  for  Pd  and  Pf  in 
terms  of  these  point  coordinates.  If  this  expression  of  the  cost  is  minimized  with 
respect  to  each  coordinate  there  will  be  as  many  equations  as  the  number  of 
coordinates.  Solving  these  equations  simultaneously  yields  the  quantizer  parameters. 
This  is  the  core  of  the  QD  algorithm  which  is  summarized  as  follows: 

1.  Derive  the  threshold  equation  of  the  CD  system. 

A  (Vj  ,\\  )  =  C  (3.3') 

2.  Approximate  the  threshold  equation  by  a  stepwise  curve  satisfying  the  X  and  M 
constraints. 

3.  Write  an  expression  for  the  cost  in  terms  of  the  curve  parameters. 

4.  Minimize  the  average  cost  with  respect  to  the  curve  parameters. 

Let  us  illustrate  how  the  algorithm  works  for  the  case  of  detection  of  a  known 
signal  in  gaussian  noise. 

3.  An  Example:  The  Known  Signal  in  Gaussian  Noise 

Consider  Figure  3.2  when  and  \\  are  given  by 

Hn  :  v.  =  n. 

H.  :  v.  =  a.  +  n.  ,i=  1,2  (3.3S) 

1  '  1  1  1 
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where  the  a/s  are  positive  constants  and  N  -  [nt  n-,  ]l  is  a  gaussian  random  vector  of 
zero  vector  mean  with  covariance  matrix: 

K  =  P  P  (3.39) 

Lp  lj 

It  is  required  to  design  the  N-level  and  the  M-level  quantizers  and  Q-, 
and  the  decision  rule  y  where 

V  :  Ylq  x  Y2q  -  (0,1)  (3.40) 

to  minimize  the  average  decision  cost. 

Procedure  following  the  QD  algorithm 

The  threshold  equation  of  the  CD  problem  has  been  shown  in  Chapter  II  to  have  the 
form: 


(a!  -P  a2  )>'i  +(a2  +P  ai  )y2  =(ai2  +a22  _P  ai  a2  )/2  +  (l-p2  )log(C).  (3.41) 


1.  The  CD  curve  is  a  straight  line  in  the  y2  plane. 

2.  Possible  stepwise  approximations  for  the  threshold  equation  are  shown  in 
Figure  3.4  .  We  notice  that  in  Figure  3.4  a  and  c  the  two  quantizers  have  the 
same  number  of  quantizer  levels.  While  in  Figure  3.4  b  and  d  one  quantizer  has 
one  more  level  than  the  other.  From  Chapter  II,  we  can  expect  that  the 
constant  C  will  decide  the  superiority  of  a  or  c  and  of  b  or  d.  We  shall 
consider  optimum  parameters  of  Figure  3.4  a  and  b.  Similar  treatment  can  be 
considered  for  Figure  3.4  c  and  d.  In  Figure  3.4  a  the  point  Xj  =  -oo  while  Tj 
is  finite.  In  Figure  3.4  b  Xj  =  -20  and  Tj  =  20  . 

3.  The  probability  of  detection  of  the  decision  rule  of  Figure  3.4  a  is  given  by 


1  1 


(3.42) 


and  Pf  is  given  by 

pf  -i)  LXi*lJr”<y,.y2  'H0)<iy1dy2. 
1  i 


(3.43) 


For  the  detection  rule  of  Figure  3.4  b  Pd  is  given  by  : 


P^1^.  $+‘C«WHi>dyidy: 


(3.44) 


and  Pf  is  given  by 


,X;j,  ,°o 


pr  =  £  ixi  +  Iir.ny,y2/H0)dyldy2 


(3.45) 


J.  Necessary  conditions  for  optimality  of  parameters  of  the  curve  in  Figure  3.4  a 


A  (T  )  =  C 


J'^VVVdy, 

_  i _ . 

f^*lfl5VT,H1)dy1 


,i=  1,2 . N 


(3.46) 


J  T;*1  fCy2/X..H0)  dy2 

A  (X;  )  =  C  -  4=  2,3 . N 

J  i*1Ky2;Xi,Hl)dy2 


(3.47) 


For  Figure  3.4  b  .the  optimality  conditions  are 


Jxi+l«y1/T,H0)dy| 

A  (T,  )  =  C  -U -  ,i=2,3 . N 


(3.4S) 


,1=2,3 . N. 


(3.49) 


V 


m  —  -  •  .F  ^ 


It'1  “y2 

A  (X;  )  -  C  — ‘ - 

«y2/Xi.H,)  dy2 

The  last  rvvo  equations  are  exactly  the  same  as  the  necessary  conditions  for 
optimizing  detection  using  a  Primary  Decision  Maker  and  its  quantized  second  opinion 
for  the  same  signals  in  gaussian  noise.  Recall  that  the  information  available  at  the 
PDM  is  more  complete  than  that  available  at  the  fusion  center  of  two  quantized 
observations.  Yet  the  two  problems  have  the  same  solution.  This  is  a  proof  of  the 
following  lemma. 

Lemma  3.2 

Optimum  detection  of  known  signal  in  gaussian  noise  using  two  quantized  observations 
of  N  and  N  +■  1  levels  is  equivalent  to  optimum  detection  using  the  first  quantized 
observation  and  the  second  continuous  observation. 

Lemma  3.2  is  applicable  to  any  case  with  a  monotonic  likelihood  ratio.  This  can  be 
easily  proved  by  writing  the  necessary  conditions  of  optimality  for  the  two 
configurations.  A  special  case  of  Lemma  3.2  is  that  of  N  =  2.  It  corresponds  to  the 
tandem  configuration  of  two  detectors  in  a  Distributed  Detection  Network  (DDN)  [10]. 
The  "downstream’'  detector  (decision  maker)  maxes  its  decision  based  on  its  own 
observation  and  the  "upstream"  detector's  decision. 

D.  NUMERICAL  SOLUTION  FOR  THE  SYSTEM  PARAMETERS 

It  is  of  interest  to  compare  the  four  sets  of  equations  {(3.24  ),(3.26)}, 
{( 3.32).( 3.33)} ,  {(3.46  ),(3.47)}  and  {(3.48), (3.49)}  with  that  of  Lloyd  and  Max  [1S.23]  for 
minimum  distortion  quantizer  parameters. 

Max's  trial  and  error  algorithm  to  solve  this  set  of  nonlinear  equations  can  be 
used.  However  Max's  algorithm  is  very  time  consuming  [24],  We  have  used  instead 
the  method  of  successive  substitutions  with  an  initial  guess  satisfying 


X2  <  X3  <  ....£  XN  (3.50) 


f  V 


and  put  the  equations  in  the  form 


The  kth  iteration  is  then  given  by 


\  "  G  (  \.i  > 


(3.52) 


We  will  devote  the  next  chapter  to  solving  some  numerical  examples  using  this  method. 

E.  SUMMARY 

In  this  chapter  the  method  of  detection  using  quantized  sensor  observations  has 
been  introduced.  This  method,  referred  to  by  QD,  can  have  significant  performance 
improvement  compared  to  the  distributed  detection  algorithm  (DD)  with  only 
marginally  more  demand  on  the  communication  channels.  The  QD  algorithm  involves 
approximating  the  CD  threshold  hyperplane  by  a  stepwise  hyperplane  that  can  be 
spanned  with  the  quantized  data  and  that  minimizes  the  detection  cost. 

Also  the  equivalence  between  two  detection  configurations,  one  with  tandem 
connection  and  the  other  with  hierarchical  structure,  has  been  shown. 
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IV.  NUMERICAL  RESULTS 


In  this  Chapter  some  examples  are  solved  numerically  using  the  QD  algorithm. 
First  the  detection  of  known  signals  in  gaussian  noise  is  considered.  Next  detection  of 
signals  with  exponential  distribution  is  considered.  Finally,  the  algorithm  will  be 
applied  to  differentiating  between  gaussian  signals  with  different  variances. 

A.  KNOWN  SIGNAL  IN  GAUSSIAN  NOISE 

Again  consider  Figure  3.2  when  yt  and  y2  are  given  bv 

Ho  :  >'i  =  ni 

Hj  :  =  a.  +  n.,i-  1,2  (4.1) 

with  a,  =  4  and  a2  =  2.  The  noise  vector 

N  =  [  nj  n2  f  (4.2) 

is  of  zero  vector  mean  and  with  covariance  matrix  given  by. 


where  p  is  given  by 

P  =  E  {  nt  n2  )  .  (4.4) 

It  is  required  to: 

1.  Design  the  primary  decision  maker  PDM  and  its  N-lcvel  Quantizer  to  minimize 
the  average  decision  cost.  We  have  designated  this  structure  configuration  A. 

2.  De  sign  the  N-level  quantizers  Q[  and  Q,  and  the  decision  rule  uo  to  minimize 
the  average  decision  cost.  We  have  designated  this  structure  configuration  B. 

3.  Compare  the  performance  of  the  two  configurations  and  that  of  the  completely 
centralized  system. 

Following  the  algorithm  we  have: 

1.  The  threshold  equation  for  the  CD  problem  given  by, 


(al  -P  a2  )yj  +(a2  -p  a{  )y2  =  (a2  +  a22  -2p  a{  a2)/2  +  (l-p2  )Iog(C)  (4.5) 


a  straight  line  in  the  yt  y2  plane. 

2.  Figures  3.4  a  and  3.4  b  show  the  decision  regions  for  configuration  A  and 
configuration  B  respectively. 

3.  Probability  of  detection  Pdl  and  probability  of  false  alarm  Pn  of  PDM  are 
given  by: 


dl 


exp(-yj2  12) 


-  a,  •  p  (v,  •a[  ) 
V  (1-P2) 


d?l 


(4.6) 


and 


v: 


n 


=  ^  f'1  +  1- —  exp(-v  2  ;2)  erfc/^ — —  -  i  dv.  . 

i=1  JXj  v  2it  -1  \  V  (1-P2)  / 


(4.7) 


Also  Pd?  and  Pj-,  of  configuration  B  are  given  by: 


P  =  ^  rxi  +  i 

d2  i=1  hi 


J  2tc 


exp(- 


y,2.2)  erfC<T 


'T,  -  a,  -  p  (y,  -a.  ) 


0-P-) 


■ }  d>l 


(4.8) 


and 


=  ^  rXi  +  i  1 

:  J  i 


P  .  ■  j  - —  exp(-v  *  2)  erfc 

c  i  =1  J  X.  V  2n  1 


1  • 


(4.9) 


1.  For  configuration  A  equations  of  the  quantizer  interval  end  points  and 
corresponding  PD  Ms  thresholds  for  the  gaussian  case  are  given  by 


A  (Xk 


=  C 


erfc  -erfcl1* 


-P  X, 


V  u-p*> 


erfc 


{ 


'P  'Xk~ai 


v7  (1-PD 


\  -  erfc  f 


■T,  -  a,  -  p  (X.  -a,  ) 


v  (1-P*) 


} 


,k=  2,3,  N 


,k=  2,3,  N.  (4.13) 


We  have  solved  the  system  of  equations  of  the  two  configurations  using  the 
method  of  successive  substitution  for  N  =  2,  3,  4,  5  and  6  .  Figure  4.1  shows  the 
receiver  operating  characteristics  ROC  for  the  two  configurations  for  p  =  0,  for  different 
values  of  N.  The  ROC  for  the  CD  system  is  also  shown.  The  effect  of  p  is  illustrated 
in  Figure  4.2.  The  figure  shows  ROC  curves  for  Configuration  A  for  different  values  of 
N  and  for  p  =  0  and  0.25.  Figure  4.3  shows  the  average  cost  of  Configuration  B  and 
CD  vs.  C,  for  different  values  of  N.  The  relation  between  the  cost  of  detection  for 
Configuration  B  vs.  the  number  of  quantization  levels  is  shown  in  Figure  4.4  .  The 


figure  shows  the  exponential  decay  of  the  detection  cost  as  the  amount  of  information 
available  at  the  fusion  center  increases. 

The  following  results  are  noted  from  the  curves. 

1.  Configuration  A  has  better  ROC  curves  than  Configuration  B.  The 
performance  difference  is  large  for  N  =  2  but  gets  smaller  as  N  increases. 

2.  Both  performances  converge  to  that  of  the  CD  in  a  uniform  manner. 

3.  As  the  correlation  coefficient  increases  the  performance  difference  decreases. 
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4.  As  N  increases  the  average  detection  cost  gets  smaller  and  tends  to  that  of  the 
CD.  Since  the  number  N  reflects  the  mutual  information  between  the  input  and 
the  output  of  the  quantizers,  the  relation  between  the  performance  degradation 
and  information  delivered  to  the  fusion  center  is  strong. 


B.  SIGNALS  WITH  EXPONENTIAL  DISTRIBUTIONS 


Consider  again  Figure  3.2  .  Let  Vj  and  y2  have  the  following  distributions: 


Hq  :  fly,  )  =  X0  exp(-  X0  yt  ) 


(4.14) 


n>'i  )  =  X,  expl.-Xj  y.  )  ,i=  1,2 


and  assume  that  A.,  is  less  than  /.Q  .  It  is  required  to  design  the  quantizers  and  fusion 
rule  that  minimize  the  average  decision  cost 
Following  the  QD  algorithm  we  have: 

1.  The  CD  threshold  equation  is  given  by 


y,  -  y,  =  C, 


(4.16) 


where  C,  is  given  bv 


i/.n  X,  i- 

C,  =  — - - —  C  . 


/•0  - 


The  CD  threshold  equation  is  a  straight  line  in  the  first  quadrant. 

2.  Figure  4  5  shows  possible  approximations  of  the  threshold  equation.  For  N  =  2  . 
the  symmetry  suggests  equal  detector  thresholds.  For  N  >  3  let  us  fix  Xt  and 
1\-  to  zero. 


3.  The  probability  of  detection  and  probability  of  false  alarm  Pd  and  Pf  are  given 
be 


X. 

’  =  '  ;  expi-X,  X  )-  expi-X,  X  .  i  :  exn-'/., 

J  ,  i  ■  .1  r  1  l  *  i  1  1 


«"*  -*  /  J  ’  .  » 


(4.19) 


N 

Pf  =  z  l  exP('^0  Xi  )•  exP(*X0  Xi  +  1  )  ]  exP(-X0  Ti  ) 
i=  1 

4.  Writing  an  expression  of  the  average  cost  in  Pd  and  Pf  as  before  and  minimizing 
with  respect  to  Xk  ,k=2,3,..N  and  Tk  ,  k=  1,2,..  N-l  one  obtains  the  set  of 
equations 


exp(-Xj  )-exp(-Xj  Xk  +  l) 


,k=  1,2,...  N-l 


(4.20) 


exp((kn  -X,  )X.  }=  —  C 


exp(-X0  Tk  )-exp(-XQ  ^  ^ 

exp(-Xj  Tk  )-exp(- XQ  Tk  +  1  )  ’ 


(4.21) 


This  set  of  equations  Have  been  solved  by  the  method  of  successive 
substitutions  for  XQ  =  2,  Xj  =1,  and  for  N=2,3,4,5  and  6.  A  FORTRAN 
program  to  calculate  the  quantizer  parameters  is  given  in  Appendix  D. 

Figure  4.6  shows  ROC  curves  for  the  quantized  as  well  as  the  CD  systems.  The 
average  detection  cost  is  shown  in  Figure  4.7. 

We  note  the  following: 

1.  The  largest  performance  improvement  occurs  when  we  switch  from  N  =  2  to  N 
=  3  (  i.e.  oniy  less  than  one  more  information  bit  per  detection). 

2.  The  performance  curves  {  ROC(N)  )  and  {  R(N)  )  converge  uniformally  to  the 
performance  of  CD 

C.  GAUSSIAN  SIGNALS  WITH  DIFFERENT  VARIANCE 

Consider  again  the  structure  of  Figure  3.2  .  Let  sensor  observations  >’j  and  y,  be 
independent,  identically  distributed  gaussian  random  variables  of  zero  mean.  However, 
under  HQ,  Varfv.  )=<r02, 
and 

under  II,.  Var(y.  )=  ,i=  1,2.  For  specificity,  let 

CTg  =  1  and  <7 1  =  v'  2  . 

Quantized  sensor  observations  are  sent  to  the  fusion  center  to  decide  which  of  the 
hypothesis  is  true.  It  is  required  to  design  the  quantizers  Qf  and  Q,  as  well  as  the 
fusion  rule  to  minimize  the  average  decision  cost. 
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Figure  4.5  Approximation  of  the  Threshold  equation  for 
Different  Values  of  N,  for  Exponential  Signals. 
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Figure  4.7  Average  Detection  Cost  for  Different 
Values  of  N  for  Exponential  Signals. 


Following  the  QD  algorithm  we  have: 

The  CD  system  decision  rule  is  a  likelihood  ratio  test.  The  CD  detector  declares 
Hj  is  true  if 

yi2  +  y22  <  (l/2)log[<r12  /(<tq2  C)]/(<xq-2  -af2  y1  (4.22) 

otherwise  it  will  declare  HQ  is  true.  The  threshold  equation  is  the  circle 

v  +y22  =  V  (4-23) 


where  RQ2  is  the  right  hand  side  of  inequality  (4.22  ). 

Possible  approximations  of  the  CD  threshold  equation  are  shown  in  Figure  4.8  . 

Figure  4.8  a  corresponds  to  3-level  quantizers.  The  corresponding  probability  of 
detection  and  probability  of  false  alarm  are  given  by; 

Pd(3)  =  [erfl-X:a1)]2  (4.24) 

and 

Pf  (3)  =  [erf(-X,  <tq)|2  (4.25) 

where  yt  and  y2  are  subdivided  by  the  points  X  and  -X.  For  the  5-level 
quantization  approximation  of  Figure  4.8  b,  the  probability  of  detection  and 
probability  of  false  alarm  are  given  by 


Pd  (5)  =  erfIX3  /(Tj  )  {2erfl[X2  /<r1  )-  erfIX3  i<tx  )  )  (4.26) 

and 

Pf(5)  =  erf(X3  ,'<Jq  )  {2erf(X2  /< rQ  )  -erf(X3  ;aQ  )  }  (4.27) 

where  X2  ,  X3  ,-X3  and  -X2  defme  the  the  quantization  intervals  of  both  yt 
and  y2  . 

Inserting  Pf  (3)  and  Pd  (3)  into  R  in  (3.3  )  and  minimizing  R  with  respect  to  X 
gives 
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A  (X  )  =  C 


(4.28) 


<*j_  erffX  /(T0  ) 

<tq  erf(X  /(T1  ) 

Also  inserting  Pd(5)  and  P^5)  into  (3.3  )  and  minimizing  R  with  respect  to  X2 
and  X3  gives; 


A(X2) 


erfIX3  /q0 ) 
<tq  erf(X3  /<t^  ) 


(4.29) 


A  (X  )  -  C  eTfIX?  /<yo  } '  erfIX3  /qo } 

"  3  <*0  erf(X2  /<T1  )  '  erf^X3  ',<T1  ) 


(4.30) 


Solution  of  these  implicit  equations  in  the  quantizer  parameters  can  be  carried 
out  by  the  method  of  successive  substitution.  The  FORTRAN  program  to  calculate 
them  for  any  value  of  <70  and  ffj  is  given  in  Appendix  F. 

"ugure  4.9  shows  the  average  detection  cost  vs.  C  for  3-level  and  5-level 
quantizer  systems.  Detection  cost  of  CD  is  also  shown.  The  figure  shows  that  the 
detection  cost  decreases  dramatically  using  5-level  quantizers  in  comparison  to  3-level 
quantizers.  The  cost  of  the  CD  system  is  only  slightly  lower  than  that  of  the  5-level 
quantizers. 

Similar  procedures  can  be  carried  out  for  the  case  of  correlated  observations.  The  CD 
curve  in  this  case  is  an  ellipse  with  principle  axes  passing  through  the  origin.  It  can  be 
approximated  in  a  similar  way  as  the  circle. 

D.  CONCLUSION 

The  above  examples  show  the  uniform  convergence  of  the  Quantized  Detection 
Algorithm  to  the  Centralized  Detection  Algorithm.  The  Distributed  Detection 
Algorithm  is  a  special  case  of  QD.  It  follows  that  Quantized  Detection  is  an  efficient 
utilization  of  bandlimited  communication  channels. 
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V.  THE  CASE  OF  VECTOR  OBSERVATIONS 


A.  INTRODUCTION 
In  the  previous  two  chapters  the  QD  problem  for  the  case  of  scalar  sensor 

observations  was  solved.  It  is  now  time  to  extend  the  QD  algorithm  to  the  case  where 
each  local  observation  is  a  vector  Y-  .  The  QD  algonthm  can  be  applied  as  long  as  the 
corresponding  sufficient  statistic  for  the  centralized  detection  problem  can  be  divided 
into  local  statistics  to  be  quantized.  Let  us  consider  the  gaussian  case  and  put  it  in  the 
previous  framework. 

B.  QUANTIZED  DETECTION  WITH  VECTOR  OBSERVATIONS 

For  the  structure  of  Figure  5.1  the  observations  at  locations  1  and  2  are  given  by 


Hn  :  Y.  =  N. 

u  — J  —l 

and 

H,  :  Y.  =  A.  +  N. 

1  —1  — 1  —l 


Let  us  denote  the  observation  vector  by  Y 


(5.1) 


(5.2) 


The  noise  vector  N  ,  given  by 


(5.3) 


is  multivariate  saussian  with  zero  vector  mean  and  covariance 


,  /?.,  and  are  the  covariance  matrices  of  the  noises  at  locations  1  and  2  and  their 
common  covariance  matrix.  The  signal  vector  A  is  given  by 

(5.5) 

The  CD  system  decides  that  7  belongs  to  Zj  if  [6] 

exp  {  (-1/2)  [  (7  -AYR'1  (7  -A)  -7  'fl'1  7  ]  }  £  C  .  (5.6) 


The  CD  threshold  equation  can  be  written  in  the  form 

A'R -1  7  =  log(C)  -( 1 ,  2)  AT?'1  A.  (5.7) 

Using  the  block  matrix  inversion  lemma  [25],  (5.7  )  can  be  written  in  the  form 

a  +  P  7,  =  log(C)  -(1/2)  AT?’1  A  .  (5.S) 

In  (5.8  )  a  and  p  are  given  by 
a  =A'j  (5t  -R[2R2'1  R2l  )‘l 

-a,  (r2  -e21  fif1  e21  r1  z2\  Bil  (59> 

and 

P  “-A'j  (Si  -E[2E2l  32i  )_1  Znh'1 

+  A  2  (E2 -521  ^j'1  ^12  r1  ■  (5  10) 


Denoting 


V,yS 


U  =  Pi-, 


(5.12) 


(5.S  )  becomes 


lj  +  1,  =  log(CM  1/2)A7?‘  A 


(5.13) 


where  lj  and  U  are  bivariate  gaussian  with  zero  vector  mean  under  hypothesis  Hc 
Under  hypothesis  H1  their  vector  mean  is 


(M""A 

y  [Pd; 


(5  14) 


Covariance  of  1.  and  1,  is  given  by 


Cov(l1  ,1,  )  = 


a  R.  a 


P'S,  P 


(5.15) 


In  (5.15  )  p  is  given  by 


p  =  a  R[2  p  v7  ^(a  R1  a  )'1  (P  fl,  p  )'{J 


(5.16) 


The  distributed  signal  processing  is  to  form  local  linear  combinations  ^  and  U  ,  then 
quantize  them  as  before.  This  processing  is  also  shown  in  Figure  5.1. 

C.  SUMMARY 

In  this  Chapter  it  is  shown  that  the  QD  algorithm  can  be  extended  to  the  case  of 
sensor  vector  observations.  An  application  is  the  case  of  high  quality  local  area 
communication  and  lower  quality  long  distance  communications.  In  this  case  sensor 
observations  in  local  areas  are  gathered  at  a  local  processor  to  form  the  local  sufficient 
statistics.  Quantized  local  statistics  are  then  sent  to  the  global  far  away  processor  for 
fusion. 
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VI.  OPTIMUM  ESTIMATION  USING  QUANTIZED  SENSOR 

OBSERVATIONS 


A.  INTRODUCTION 

In  the  previous  part  of  this  thesis  there  are  situations  in  which  a  group  of 
observers  make  local  decisions  that,  taken  in  combination  determine  the  overall 
performance  of  a  system.  The  observers  may  or  may  not  be  interconnected.  However, 
even  when  they  are,  for  a  variety  of  considerations  such  as  limitations  on 
communications  bandwidth,  transmitter  power, security,  or  perhaps  the  very  nature  of 
the  observers  themselves,  only  decisions  may  be  interchanged  between  them  and  not  ail 
the  observations  upon  which  their  decisions  are  based  [1,5,26-33]. 

Another  case  of  interest  concerns  the  encoding  of  high  resolution  measurements 
for  transmission  between  observers  using  a  small  number  of  bits.  Here  a  remote 
observer  must  decide  which  of  N  possible  discrete  values  best  represents  his 
observation.  A  second  observer  is  to  combine  his  local  observations  with  the  discrete 
data  from  the  first  in  an  optimum  manner.  In  this  chapter  we  consider  the  problem  of 
regeneration  of  a  remote  sensor  observation  using  its  quantized  representation  and  a 
local  observation.  The  design  of  the  quantizer  at  the  remote  sensor  location  and  the 
optimum  linear  estimator  to  combine  the  quantized  data  with  the  local  observation  to 
minimize  the  expected  mean  square  estimation  error  will  be  considered.  Generalization 
of  the  results  to  the  vector  case  is  also  shown. 

B.  THE  LINEAR  MINIMUM  MEAN-SQUARE  ESTIMATE  OF  Y  , 

Consider  the  structure  of  Figure  6.1  in  which  the  observation  is  quantized 
into  Vj  by  a  quantization  rule  y 

y  :  yL  -  Yiq-  on 

The  quantized  data  Vj  is  sent  to  sensor  S-,  site. 

The  linear  minimum  mean  square  estimate  of  the  observation  y(  from  and  y,  is 
shown  in  (Appendix  F)  to  be 


where  we  have  assumed  that  y(  and  y,  are  random  variables  with  zero  mean  and 


variances 

E{y-  }  =  (T'j  .i*  1.2  (6.3) 

and  correlation 

P  =  E ( y i  y;  }.al  <t2  (6.4) 

The  scalar  quantities  t]  and  ji  are  parameters  of  the  quantizer  and  are  given  by 

(6.j) 

and 

"  <6-6’ 


N  is  the  number  of  quantization  levels  and  Ck  and  Pk  are  given  by 


Qk  is  the  k^  quantization  value  a.-.d 

,k=  1,2 . X,  are  the  quantization  inter.  .  r 

X!  =-X  and  \\  .  .  -  / 
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The  expected  mean  square  error  is  given  by  [34] 


-ylq  Y  }  =  <Tt2  -E{yi  Z1  }E{2  xl  E{7  yj  } 


(6.9) 


where  I  isthe  vector 


Equation  (6.9  )  can  be  written  in  the  form  (Appendix  D) 


(6.10) 


E{e2  }  =  ffj2  (1-p2  )(l-to  )/(l*p2(o) 


(6.11) 


where  to  is  given  bv: 


to  =r  /n- 


(6.12) 


A  plot  of  E(e2};<r12  vs.  to  is  shown  in  Figure  6.2  for  p2  =  0,  0.25,  0.5,  0.75.  The  figure 
shows  that  the  mean  square  error  is  decreasing  with  to.  Recall  that  the  criterion  is  to 
minimize  the  mean  square  error. 

Equivalently  the  problem  now  is  to  maximize  to  over  all  quantization  rules  where 


i 


to  = 


(6.13) 


Appling  the  Cauchy  Inequality  [35]  to  the  numerator  yields 


( £  Pt  Qk  ck  )J  sk(^  pk  Qk2  \(£,  pk  ck2 ) 


r=  i 


(6.14) 


with  equality  if  and  only  if 


Q*  = 


(6.15) 
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Therefore 


<a  £ 


1 


l 

r=  1 


pkQk 


2 


(6.16) 


gives  an  upper  bound  of  <o.  Equation  (6.15)  says  we  maximize  <o  ,  and  thus  minimize 
E{e2}  by  making  the  quantization  level  Qk  equal  to  the  conditional  mean  of  yt  given 
that  yt  lies  in  the  kth  quantization  interval.  This  is  one  of  the  conditions 
characterizing  the  classical  Lloyd-Max  quantizer  [18,23.}  There  remains  the  problem  of 
how  to  pick  Xk  ,k  =  1,2,...N,  so  that  the  upper  bound  of  co  in  (6.16)  is  maximum. 
Notice  that  the  upper  bound  of  <o  is  q.  Therefore,  the  optimum  quantizer  will  be  a 
Lloyd-Max  quantizer  if  we  prove  that  maximizing  q  over  all  choices  of  the  set  of 
points  {Xk},  k=  1,2, ...N,  is  equivalent  to  minimizing  the  distortion  E{(y]-y1^)2}.  Since 


E((yrylq)2}  =  <r12-2o,i2Ti  +  <T12  q 

=  (tj^i-n) 


(6.17) 


then  maximizing  q  will  minimize  the  distortion  E{(y1-ylq)2}  and  vice  versa.  Since  the 
Lloyd-Max  quantizer  is  the  optimum  quantizer  for  minimum  distortion  it  follows  that 
it  is  also  optimum  for  our  problem.  Accordingly  choose  Xk 's  such  that  [23,18],  (see 
also  Appendix  G) 


Xk  = 


Qk  +Qk. 


k-I 


,k=  1,2,. ..N. 


(6.18) 


Equations  (6.15)  and  (6. IS)  along  with  (6.7)  completely  design  the  quantizer  [23,18]. 
Parameters  of  the  Lloyd-Max  quantizer  can  be  calculated  efficiently  using  the  method 
of  successive  substitution  (Appendix  G).  Values  of  E(e2  J/ffj2  vs.  N  are  listed  in  Table 
2  for  p  =0,  0.25,  0.5,  0.75.  The  table  shows  the  exponential  decay  of  the  MMSE  as  the 
number  of  quantization  levels  increases. 

Table  3  shows  a  comparison  between  the  average  number  of  bits  per  sample  used  in 
this  system  and  another  method  in  which  the  Maximum  Output  Entropy  (MOE) 
Quantizer  [36]  is  used.  Huffman  coding  [37]  is  assumed  for  both  quantizers. 


TABLE  2 

MINIMUM  MEAN  SQUARE  ERROR  VS.  THE 
NUMBER  OF  QUANTIZATION  LEVELS 


0 

0.25 

0.5 

0.3634 

0.3548 

0.3241 

0.1175 

0.1166 

0.1131 

0.0345 

0.0345 

0.0342 

0.0095 

0.0095 

0.0095 

0.0025 

0.0025 

0.0025 

0.0006 

0.0006 

0.0006 

0.00016 

0.00016 

0.00016 

0.75 


0.2477 


0.1021 


0.0330 


0.0094 


0.0025 


0.0006 


0.00016 


TABLE  3 

COMPARISON  OF  THE  AVERAGE  NUMBER  OF  BITS  IN 
THE  MMSE  AND  THE  MOE  SYSTEMS 


C.  CONCLUSION 

The  trade  off  between  performance  and  communication  is  clear  from  Table  2. 
For  p  *»0.5  the  relative  MMSE  is  0.75  without  communication.  This  corresponds  to 
substituting  to  =  0  in  (6.11  ).  The  relative  MMSE  decreases  to  0.32  using  one 
information  bit  per  sample.  The  relative  MMSE  is  0.11  using  two  bits/sample.  It  is 
only  0.03  using  3  information  bits/sample  (N=8)  and  is  0.00016  using  7  bits/sample. 
We  also  notice  that  for  high  number  of  quantization  levels  the  estimation  error  is 
approximately  equal  to  the  the  quantization  error.  This  means  that  the  estimator 
depends  mainly  on  y^  for  fine  quantization.  For  coarser  quantization  the  estimator 
depends  heavily  on  y2  to  reduce  the  MMSE.  Table  3  shows  that  the  designed  system 
has  considerable  reduction  in  the  number  of  bits  per  sample  compared  to  the  MOE 
quantizer  system. 


D.  GENERALIZATION  TO  THE  VECTOR  CASE 

In  this  section  we  will  consider  regeneration  of  a  random  vector  Yj  from  its 
quantized  version  Yt  and  a  correlated  continuous  scalar  y2  .  As  an  application 
consider  a  sensor  S2  monitoring  the  activities  of  N  stations.  Due  to  some 
considerations,  perhaps  of  safety  nature,  only  simple  sensors  can  be  placed  near  the 
stations.  Because  of  other  considerations,  such  as  limited  bandwidth  communication 
channels,  only  quantized  sensor  measurements  can  be  sent  to  the  monitor.  Specific 
examples  can  be  the  case  of  monitoring  the  states  of  a  target  in  a  far  field  or  the 
positions  of  N  targets  in  a  multitracking  problem  [38,39].  Another  example  is  to 
monitor  the  radiation  levels  outside  of  N  nuclear  reactors.  A  third  example  is 
monitoring  the  activities  of  N  enemy  transmitters. 

Let  us  design  the  quantizers  at  the  N  sensor  sites  and  the  estimation  rule  at  the 
monitor  site  so  as  to  minimize  the  mean  square  error  of  each  component  of  Y  .  Let  Yj 
,the  sensor  observation  vector  be  given  by; 


*i  =t  >'ii  y 


12 


(6.19) 


where  Vj.  is  the  sensor  observation  ,j=  1,2,...,N.  We  will  assume  that  components  of 
7  are  independent,  i.e. 


flyn  /yu  .y2)=fb’i  /y2 )  .i  *  i.  1.2 . n. 


(6.20) 


Under  the  above  conditions,  also  yi;  and  y^  are  conditionally  independent  for  i  *  j,  so 
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The  MMS  estimate  of  Jj  given  is  given  by 


Xi  -E  {  Xi  /Zlq  .y2  >  -  (6.22) 


Or 


LE'{yIN  /yINq  ,y2 


Let  us  denote  the  error  vector  by  E  ,  so 


(6.23) 


I  =  [  el  e2  ...eN  ]‘ 


(6.24) 


where  e;  is  the  error  in  estimating  y.  ,i=  1,2,...,N.  The  VIMS  error  covariance  matrix  is 


E {EE1}  •E{(Zl  -Tj  )1}. 


(6.25) 


The  trace  of  the  error  covariance  matrix  is  given  by 


trace(E(Er})=I  E(e.2  } 

i=  1  1 


where 


(6.26) 


e. 

1 


=yrE^n  />’uq  .y2> 


(6.27) 


Minimizing  the  trace  of  the  covariance  matrix  in  (6.26  )  is  accomplished  by  minimizing 
each  summand  alone  since  every  summand  is  nonnegative.  Now  assuming  Linear 
Minimum  Vlean  Square  Estimation,  the  problem  of  minimizing  E  (a2  }  implies  using 
the  Lloyd- Vlax  Quantizer  to  quantize  yu  as  was  shown  previously. 


VII.  SUMMARY,  RESULTS,  CONCLUSIONS  AND  SUGGESTIONS  FOR 

FUTURE  RESEARCHS 


A.  SUMMARY 

This  Thesis  begins  by  listing  some  reasons  why  Distributed  Signal  Processing  is 
more  practical  than  Centralized  Signal  Processing.  The  status  of  Distributed  Detection, 
an  important  case  of  Distributed  Signal  Processing,  and  its  complexity  are  reviewed  in 
Chapter  I. 

Chapter  II  deals  with  the  problem  of  optimum  fusion  of  local  decisions  into  a 
global  decision.  The  relationship  between  the  optimum  fusion  rule  and  the  ratio  of 
error  cost  is  shown.  The  dependance  of  the  performance  of  the  optimum  fusion  rule  on 
the  correlation  coefficients  between  sensor  observations  is  throughly  analyzed.  For 
higher  values  of  the  correlation  coefficients  the  Distributed  Detection  system  is  shown 
to  reduce  to  the  detector  of  the  highest  signal- to-noise  ratio. 

A  compromise  technique  between  Centralized  Detection  and  Distributed 
Detection,  Quantized  Detection,  is  suggested  in  Chapter  III.  The  main  issue  of  that 
chapter  is  to  control  the  degree  of  centralization  according  to  the  communications 
channel  constraints.  The  Quantized  Detection  technique  replaces  local  detectors  by 
quantizers  and  sets  a  global  fusion  rule  that  approximates  the  centralized  decision  rule. 
The  algorithm  matches  the  other  techniques  at  extreme  limits. 

Chapter  IV  contains  some  specific  applications  of  the  Quantized  Detection 
Algorithm  for  detection  problems.  A  significant  performance  improvement  is  attained 
by  replacing  Distributed  Detection  with  Quantized  Detection  with  three  quantization 
levels  (one  and  half  information  bits  per  sample  vice  one  information  bit  per  sample). 

Chapter  V  considers  applicability  of  the  Quantized  Detection  Algorithm  to  the 
case  of  vector  observations.  In  this  case  local  sufficient  statistics  are  quantized  in  the 
same  way  as  before. 

Chapter  VI  deals  with  the  regeneration  of  sensor  observations  from  their 
quantized  versions  and  another  correlated  observation.  The  local  quantizers  and  the 
optimum  linear  data  fusion  are  designed.  We  arrive  at  the  following  results  and 
conclusions. 


B.  RESULTS 

1.  Detection  with  Distributed  Sensors 

a.  Optimum  Fusion  Rules  in  Distributed  Detection 

The  optimum  fusion  rule  depends  on  the  ratio  of  costs  of  different  types  of 
detection  errors.  For  high  cost  of  false  alarm  relative  to  the  cost  of  missing  the  target 
the  AND  fusion  rule  is  better  than  the  OR  fusion  rule,  and  vice  versa. 

The  performance  of  the  optimum  fusion  rule  depends  on  the  degree  of 
correlation  between  sensors.  The  performance  degrades  as  the  correlation  coefficient 
increases.  The  worst  performance  of  the  optimum  fusion  rule  is  at  and  above  a  critical 
value  of  the  correlation  coefficient  p^  .  In  that  region  of  correlation  the  best  system 
employs  only  the  detector  of  higher  signal-to-noise  ratio,  ignoring  the  lower  signal-to- 
noise  ratio  sensor  entirely.  The  performance  of  the  Distributed  Detection  system 
improves  as  the  signal-to-noise  ratio  imbalance  between  sensors  increases.  However 
there  is  a  large  performance  difference  between  the  Centralized  Detection  and  the 
Distributed  Detection  for  values  of  the  correlation  coefficient  above  pcf. 

Below  pcr  the  performance  of  the  Distributed  Detection  system  improves  as 
the  correlation  coefficient  gets  smaller.  The  best  performance  (lowest  detection  cost)  of 
the  Distributed  Detection  system  is  achieved  at  p  =  -l.  Recall  that  the  Centralized 
Detection  system  has  perfect  detection  at  p  =-l.  This  is  due  to  the  efficient  use  of  the 
information  contained  in  two  observations  of  positive  signals  and  anticorrelated  noise 
samples. 

The  large  performance  difference  between  Centralized  Detection  and 
Distributed  Detection  systems  is  due  to  the  loss  of  information  in  the  local  detection 
processes.  As  a  remedy  to  the  performance  degradation  in  Distributed  Detection  we 
have  introduced  the  Quantized  Detection  Algorithm. 

b.  Quantized  Detection 

There  is  a  great  improvement  in  the  system  performance  using  Quantized 
Detection  with  three  quantization  levels  in  comparison  to  Distributed  Detection.  This 
performance  difference  between  Quantized  Detection  and  Distributed  Detection 
decreases  as  the  correlation  between  sensors  increases. 

The  Quantized  Detection  algorithm  is  applicable  to  the  case  of  vector 
observations  and  waveform  observations.  In  those  cases,  the  local  sufficient  statistics 
are  to  be  quantized  at  the  local  processor  and  transmitted  to  the  central  site  for  fusion. 


The  Quantized  Detection  algorithm  is  implemented  by  the  quantizers  as 
local  processors  and  a  fusion  rule,  suggested  by  the  Centralized  Detection  decision  rule, 
at  the  central  site.  The  quantizers  used  in  the  Quantized  Detection  algorithm  are 
designed  to  minimize  the  detection  cost. 

2.  Minimum  Mean  Square  Estimation  in  Distributed  Sensor  Systems 

Minimum  mean  square  estimation  in  Distributed  Sensor  Systems  involves  the 
classical  Lloyd-Max  minimum  distortion  quantizers  at  the  local  levels  and  linear 
processing  at  the  global  central  level.  A  faster  iterative  algorithm  to  calculate  the 
Lloyd-Max  quantizer  parameters  is  the  method  of  successive  substitution.  It  also  has 
more  accurate  results  than  previously  reported  techniques. 

C.  CONCLUSIONS 

We  conclude  the  following: 

1.  Global  optimization  of  the  Distributed  Detection  implies  picking  the  fusion  rule 
and  corresponding  local  decisions  that  minimizes  the  detection  cost. 

2.  The  optimum  fusion  rule  in  Decentralized  Detection  depends  on  the  correlation 
coefficient  ,  the  a  priori  probabilities  and  the  ratio  of  costs. 

3.  For  optimum  fusion  of  two  local  unbalanced  decisions  there  is  a  particular 
value  of  p  that  decides  the  optimum  fusion  rule. 

4.  For  p  ^  pcr  OR  fusion  is  better  for  higher  cost  of  missing  target  while  AND 
fusion  is  better  for  higher  costs  of  false  alarm. 

5.  For  p  >  pcf  the  optimum  fusion  rule  is  to  ignore  the  sensor  of  lower  signal-to- 
noise  ratio  and  optimize  the  decision  of  the  higher  signal-to-noise  ratio  sensor. 

6.  The  poor  performance  of  Distributed  Detection  compared  to  Centralized 

Detection  is  due  to  the  loss  of  information  at  the  local  levels. 

7.  The  Quantized  Detection  system  matches  the  Distributed  Detection  system  and 

the  Centralized  Detection  system  for  the  two  extreme  conditions  of 

quantization.  As  the  number  of  quantization  levels  increases  the  Quantized 
Detection  converges  to  Centralized  Detection. 

8.  The  Quantized  Detection  algorithm  has  a  tremendous  improvement  m 

performance  over  Distributed  Detection  even  with  only  3  quantization  levels. 

9.  The  performance  difference  between  Quantized  Detection  and  Distributed 

Detection  increases  as  the  correlation  of  the  observations  gets  smaller. 

10.  In  case  of  linear  Centralized  Detection  threshold  equations  in  the  observation 
space,  Distributed  Detection  and  Centralized  Detection  are  special  cases  of 
Quantized  Detection. 

11.  The  Quantized  Detection  algorithm  can  get  the  maximum  allowable 

performance  in  the  presence  of  communication  constraints. 
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,2.  The  Quantized  Detection  algorithm  can  be  applied  to  arbitrary  distributions  for 
the  observations. 

,3.  The  method  of  successive  substitution  is  applicable  to  the  design  of  many  types 
of  quantizers.  It  has  a  simple  programming  procedure  and  very  accurate  results. 

SUGGESTIONS  FOR  FUTURE  RESEARCH 

The  following  are  some  areas  the  Quantized  Detection  algorithm  can  extend  to: 

1.  Optimum  detection  using  quantized  sensor  observations  for  the  case  of 
unknown  signal  in  noise. 

2.  Detection  of  M-ary  phenomena  using  quantized  sensor  data. 

3.  Utilizing  the  Quantized  Detection  algorithm  over  noisy  channels. 

4.  Illustration  of  the  relation  between  the  complexity  in  some  suitable  units  and 
the  amount  of  information  delivered  to  the  fusion  center. 

5.  Utilizing  the  Quantized  Detection  algorithm  to  meet  the  Neyman-Pearson 
criterion. 

6.  Extension  of  Distributed  Detection  and  Quantized  Detection  to  more  than  two 
sensors  with  correlated  observations  and  unequal  SNR's. 

7.  Development  of  general  principles  for  parsing  fusion  rules  given  a  Centralized 
Detection  surface  in  N-dimensional  space. 

8.  Application  of  the  Quantized  Detection  method  to  target  detection, classification 
and  trackir  g  using  distributed  sensors. 


APPENDIX  A 

PROBABILITY  OF  DETECTION  AND  PROBABILITY  OF  FALSE 
ALARM  OF  THE  PRIMARY  DECISION  MAKER 

Given  that  the  Primary  Derision  Maker  (PDM)  receives  Qj  (the  quantization 
level  of  the  Consultant  observation  y^,  and  that  its  own  observation  is  y2,  its 
observation  space  is  divided  into  two  derision  subspaces  and  ZQy  Let  us  denote  the 
conditional  probability  of  detection  and  probability  of  false  alarm  given  Qj  by  Pdj  and 
Pfj  respectively.  Pdj  and  Pfj  are  given  by: 

Pdj  =  Pr( Declare  Hj  /ylq  =  Qj  ,Hj  is  true)  (A.l) 

and 


P^  =  Pr( Declare  Hj  /ylq  =  Qj  ,HQ  is  true). 
These  can  be  expressed  as: 


or  equivalently  as  , 

pdj  -1  . 7  J  v  -y2  />'iq  -  Qj  -Hi  )dyi  dy2 

y2  e  Alj  yl 

and 

pn  =[  .  7  I  «yi  -y2 '>'iq  -Qj  -Ho  )dyi  dy2 

y2  e  ^lj  yi 


(A. 2) 


(A-3) 


(A.4) 


(A. 5) 


(A. 6) 
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But  ffy  ,y2  /Q.  ,R  )  is  given  by  [40] 


fty  J2  /Qj  -Hi  )  = 


{ 


ffy  ,y2  /R  J/PrtQ.  /R  )  ,7l  6  [Xj  ,Xj  +  1  ] 
0  otherwise 


(A.  7) 


where  Pr{Qj  /Hi  )  is  the  probability  of  the  quantization  level  of  y1  under  hypothesis 
R  .  It  is  given  by: 


Pr(Q:/H.)  =  J  J+ify/R)  dYl  ,j-  1.2....N  ,i-0,l 
Xi 


(A. 3) 


The  probability  of  detection  and  probability  of  false  alarm  are  now 


P 


d 


Pr(Qj  /«!  )Pdj 


(A. 9) 


and 


X 


Pf'I,  IMQj /H0  )P, 


(A.  10) 


Inserting  (A. 5  and  (A. 6  )  into  {A. 9  )  and  (A.  10  )  yields 


n:yry2  'Hi )  d>'i  ^ 

Ij 


(3.4) 


and 


Pf-?  jXi+if 
f  i^l  JX. 


00 

pvy2  /h0  )  dyi  dy2 

>2  6  ^lj 


(3.5) 


where  Zl-  is  the  decision  region  Zj  given  that  yt  e  [  Xj  ,X.  +  t  ]  . 


appendix  b 

PROGRAM  LISTING  TO  CALCULATE  PARAMETERS  OF  THE  N  AND 
THE  (N  +  1)-LEVEL  QUANTIZERS 

In  this  appendix  we  give  a  program  listing  to  calculate  the  parameters  of  the  N 

and  the  (N  +  l)-level  quantizers  in  configuration  A 

£******************************************************************* 

C  THIS  PROGRAM  CALCULATES  THE  OPTIMUM  N-LEVEL  AND  (N+l) -level 

r  0TTAMTT7PP  PAPAMTTTRP^ 

C  FOR  A  SYSTEM  OF  TWO  QUANTIZERS  AND  THEIR  FUSION  CENTER. 

C  THE  RATIO  OF  COSTS  C  RANGES  FROM  0.1  AND  10  . 

C  THE  VALUE  OF  N  =2,3,4, 5  AND  6. 

C  THE  PROGRAM  USES  THE  MODIFIED  METHOD  OF  SUCCESSIVE  SUBSTITUTIONS  . 
£******************************************************************* 

Q  ****************************** 

REAL*8  X(9) ,T(9) ,XX(9) ,TT(9 ) , C , SI ,S2 , A, B ,A12 , A21 , R, Cl (20) ,R1(2) 

1 , AERR, RERR , ERROR, PD ,PF,PDl(20,9,6) ,PF1(20,9,6) ,PD2 

I , PF2 , PDC ,PFC,X2 , R3 (20 , 2 , 6 ) , Xll , DCADER, FI , F2 , APD, APF , A1 , A2 , AP 
INTEGER  K,N,I,P,N1,N2,M, J,MX 

EXTERNAL  F1,F2 
COMMON  Xll ,R 

DATA  Cl/ 10 . 0D0 , 9 . 0D0 , 8 . 0D0 , 7 . 0D0 , 6 . 0D0 , 5 . 0D0 , 4 . 0D0 , 3 . 0D0 , 2 . 0D0 , 

II. 50D0, 

11.0D0, .90D0, .80D0, .70D0, .60D0, .50D0, .40D0, . 30D0 , . 20D0 , 0 . 10D0/ 

Al=l . 0D0 
A2=2.0D0 
Sl=l . 0D0 
S2=1.0D0 
R1 ( 1 )=0 . 600D0 
R1(2)=0.8000D0 
AERR=0 . ODO 
RERR=0 . 00010D0 

C  INITIAL  GUESS 

XX(1)=-9.50000D0 
XX(2)=-1 .9OOOD0 
XX  3)=-1.50000D0 
XX(4)=-0.89000D0 
XX(5)=00 . 50000D0 
XX(6)=0.09000DO 
XX(7 1=00 . 99000D0 
XX(8 )=01 . 50000D0 
XX(9)=2.09000D0 
C  TT(1)=4.67000D0 

C  TT(2)=3 . 89000D0 

C  TT(3 )=2 . 67000D0 

C  TT(4 )=2 . 29000D0 

C  TT(5)=1 .90000D0 

C  TT(6)=1 .49000D0 

C  TT(7 )=1 , 09000DO 

C  TT(8)=00.50000D0 

C  TT(9)=0 . 19000D0 

Q********** ************************** ******************** 

C  THE  FOLLOWING  INITIAL  VALUES  OF  T'S  CORRESPOND  TO  THE  CASE 
C  OF  CORRELATION  COEFFICIENT  GREATER  THAN  A1/A2 
TT(9)=4.67000DO 
TT(8)=3 . 89000D0 
TT(7 )=2 . 67000DO 
TT ( 6 )=2 . 29000D0 
TT(5)=1 .9000000 
TT(4)=1 .49000D0 
TT(3)=1 .09000DQ 


TT{2)=00. 


, 50000D0 

TT( 1 )=Q . 19000D0 
DO  500  N=2,2 
WRITE(9,60)N 
DO  600  1=6,9 

c  INPUT  VALUE (S)  OF  THR  CORRELATION  COEFFICIENTS 

R=0 . 10D0*DFL0AT ( I ) 

WRITE (9 , 61 )R 
C  DO  200  M=1 ,20 

M=L 

C=C1(M) 

WRITE (9, 61) C 

q  ■k  -k  "k  "k  9c  i*  *  *  ★  "k  k  k  k  k  k  k  k  k  k  k  k  k  k  k  *  k  *  k  k  k  k  k  k  k  *  *  k  *  k 

c  INPUT  MAXIMUM  NUMBER  OF  ITERATIONS 

MX=250 
K=0 


HERE 


K=K+1 
IF 
IF 
IF 
IF 
IF 


.GT.  MX) 
. GT . 1 )  X 
. GT . 1 )  T 
•GT.  1)X 
.GT.  1  )T 


A=(S1*XX(2)-R*S2 


32*TT(1) )/ (S1*S2*DSQRT(  i 
A12=(S2*A1-R*S1*A2  /(S1*S2*DSQRT( 


[ { 1 . 0D0-R**2 ) *2 . 0D0 ) ) 

,  _  _ _  ,((1.0D0-R**2)*2.0D0j 

A21=(Sl*A2-R*S2*Al)/(Sl*S2*DSQRT( (1 .0D0-R**2)*2.0D0) 
TT(1)=(A2*A2/2.0DO+DLOG(C)+DLOG((DERFC(A)- 
2 . 0D0 ) / ( DERFC ( A- A12 ) -2 . ODD) ) )/A2 
T(1)=TT(1) 

A=(S1*TT(2)-R*S2*XX(2))/(S1*S2*DSQRT((1.0D0-R**2)*2.0D0)) 
B= (S2*T ( 1 ) -R*S1*XX ( 2 ) ) / ( S2*S1*DSQRT ( ( 1 . 0DQ-R**2 ) "2 . 0D0 ) ) 
XX(2)=(A1*A1/2.0DO+DLOG(C)+DLOG((DERFC(A)- 
DERFC(B) )/ (DERFC (A-A21) -DERFC ( B-A21 ) ) ) )/Al 
X(2)=XX(2) 


IF 

DO 


(N  .EQ.  2)  GO  TO  17 
15  P=2,N-1 

X(P)=XX(P) 

A=(S1*X 


P)-R*S2*T(P))/(S1*S2*DSQRT((1.0D0-R**2)*2.0D0)) 
B=(S2*X(P+1 )  -R*S1*T(P)  )/  (S2*S1*DSQRT(  (1 . 0D0-FJ  '  ^ - 


15 

17 


CONTINUE 
CONTINUE 
TT(N)=(A2*A2/2 


TT ( P ) = ( A2*A2/ 2 . ODO+DLOG ( C ) +DLOG ( ( DERFC (A) - 
DERFC(B) )/ (DERFC(A-A12)-DERFC(B-A12) ) ) )/A2 
T(P)=TT(P) 

A=(S1*T(P+1)-R*S2*X(P+1))/(S1*S2*DSQRT((1.0D0 

B=(S2*T(P)-~^-'J‘ - -  -  -  - - " 

XX(P+1)=  "■ 

DERFC(B) )/ 


-R’C*2)*2.0D0) ) 


R**2)*2.0D0)) 
R**2)*2.0D0) ) 


1))/ 

Q-kjctrk-k-k-rrk-k 

C  CHECKING  THE  ACCURACY 
C  INPUT  REQUIRED  PRECISION 
AP=0 . 10d-07 
IF( (DABS(T(N)-TT(N) ) 
1  AP ) )  GO  TO 

10  CONTINUE 

X(1 )=-10 . 0D0 
X(N+1)=10 . 0D0 
APD=0.0D0 
APF=0.0D0 
DO  81  Q=1,N 
A=X(Q) 

B=X(Q+1 
X11=T(Q 
APD=APD+0 


ODO+DLOG(C)+DLOG(DERFC( (XX(N)-R*TT(N) )/ 
1.0DO-R**2)*2.0DO) ) 

(XX(N)-A1-R*(TT(N  -A2))/ 

1 .0D0-R**2):*'2 .  0D0) ) 


1DSQRT(  i 
1/DERFC( 

1DSQRT ( i 

"  ' .  A2 

x** **************************************************** 


HERE 

.GT.  AP) .OR. (DABS (X(N) -XX(N) )  .GT. 


50D0*DCADRE ( FI , A , B , AERR , RERR , ERROR , IRE1 ) 
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IF  (IRE1  .NE.  0)  WRITE (6,60) IRE1 

APF=APF+0 . 50D0*DCADRE ( F2 , A , B , AERR , RERR , ERROR , IRE1 ) 
IF  (IRE1  .NE.  0)  WRITE(6,60)IRE1 
CONTINUE 


************** 


PD1 (M, I ,N)=APD 
PF1 (M, I ,N)=APF 

R3(M . I ,N)=1 .0D0+C1 (M)*APF-APD 

•k4rx’kyrky<'k'kiK7fk*’k-k-k-k*'Ki<7ckii*'k'k'k,k'k 


j***-k‘k'k7rkj'*1rk'k'k'k'k'k’k’kjck 


QUANTIZER  PARAMETERS 
DO  120  J=1 ,N 


120 

WRITE ( 
CONTINUE 

C 

WRITE (9, 90) 

C200 

CONTINUE 

C 

WRITE 

[9, 90) 

C 

WRITE 

9,90) 

600 

CONTINUE 

C 

WRITE 

9,90 

C 

WRITE 

9,90) 

C 

WRITE 

9,90 

500 

CONTINUE 

DC 

501 

DO  502  1=1.20 

£*** A******************** ********** ************************* 

C  OUTPUT  DETECTION  COST 


£*****  ************** 


WRITE(10,61)C1(I),{  R3(I,J.N)  ,N=2.6) 


502 

501 

60 

C90 

61 

62 


OUTPUT  PROBABILITY  OF  DETECTION  AND  PROBABILITY  OF 
FALSE  ALARM  FOR  DIFFERENT  VALUES  OF  N 

WRITE (8, 61) (  PD1(I , J,N) ,PF1(I, J,N)  ,N=2,6) 
CONTINUE 
CONTINUE 

FORMAT (IX, 14 ,4( IX, F15 . 8) ) 

FORMAT (2X  1  CON  A  ******^****************************** 1 ) 
FORMAT*  IX, 10(1X,F6.4)) 

FORMAT ( IX, 2 ( IX , F15 . 8) ) 

STOP 

END 

FUNCTION  FI (X) 

REAL*8  X,F1,A1,A2,R,X11,F11,F12 
COMMON  Xll , R 
Al=l . 0D0 
A2=2 . 0D0 

F11=DEXP (- (X-Al )**2/2 . 0D0)/ 

1DSQRT ( 3 . 0D0*DATAN{ 1 . 0D0 ) ) 

F12=DERFC( (X11-A2- (X-Al ) *R)/ 

1(DSQRT(2.0DO*(1.0DO-R**2))J) 

F1=F11*F12 


*(1.0D0-R**2)); 


RETURN 

END 

FUNCTION  F2(X) 

REAL*8  X,F2,R,X11,F11,F12 
COMMON  Xll , R 
F11=DEXP(-X**2/2.0D0)/ 
1DSQRT ( 8 . QD0*DATAN ( 1 . 0D0 ) ) 
F12=DERFC((X11-X*R)/ 
1DSQRT(2 .  QD0*( 1 . 0D0-R**2) ) ) 
F2=F11*F12 
RETURN 
END 


APPENDIX  C 

PROGRAM  LISTING  TO  CALCULATE  PARAMETERS  OF  THE  TWO 

QUANTIZERS 


In  this  appendix  we  give  a  program  listing  to  calculate  the  parameters  of  the  two 
N-level  quantizers  in  Configuration  B. 

£***★*************★★************* A***********************^^********* 

C  THIS  PROGRAM  CALCULATES  THE  OPTIMUM  N-LEVEL  QUANTIZER  PARAMETERS 
C  FOR  A  SYSTEM  OF  TWO  QUANTIZERS  AND  THEIR  FUSION  CENTER  . 

C  THE  CORRELATION  COEFFICIENT  IS  ASSUMED  TO  BE  LESS  THAN  A1/A2. 

C  FOR  THE  CORRELATION  COEFFICIENT  IS  GREATER  THAN  A1/A2  THE 
C  THE  PROGRAM  NEEDS  SLITE  MODIFICATIONS  ACCORDING  TO  THE  QD  ALGORITHM. 

C  THE  RATIO  OF  COSTS  C  RANGES  FROM  0.1  AND  10. 

C  THE  VALUE  OF  N  =2,3, 4, 5  AND  6. 

C  THE  PROGRAM  USES  THE  MODIFIED  METHOD  OF  SUCCESSIVE  SUBSTITUTIONS  . 

C**X***K*********X********** X* ****** *********** ********************* 

REAL *8  X(8) ,T(8) ,XX(8),TT(8) , A1 , A2 , SI , S2 , R,T12 ,T21 , AA2 , A,B 
1 ,AERR,RERR, ERROR ,PD,PF,PD1(20,9,6) ,PF1(20,9,6) ,PD2 
1 ,PF2 ,PDC , PFC , Cl (20 ) , A21 , A12 , C , X2 

I ,  XI 1 , DCADER , FI , F2 , APD , APF , R3 ( 20 , 2 , 6 ) 

INTEGER  K , N, I , IER1 , IER2 ,M,P ,Q ,L 
EXTERNAL  F1,F2 

DATA  Cl/10 . 0D0 , 9 .ODO , 8 . ODO , 7 . ODO , 6 . 0D0 , 5 .ODO ,4 .ODO , 3 .ODO, 2 . 0D0 , 

II. 50D0, 

11. ODO, . 90D0 , .80D0 , .70D0 , . 60D0 , . 50D0 , .40D0, .30D0, .20D0,0.10D0/ 
COMMON  XI 1 , R 

£A AAAAAAAAAAAAAAAxAAAAAAAA AA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA 

C  INPIT  SIGNALS  HERE 
Al=4 . ODO 
A2=2 . ODO 

C  INPIT  VARIANCE  HERE 
Sl=l . ODO 
S2=l .ODO 

c************************************************************** 

C  INITIAL  GUESS  OF  THE  PARAMETERS 
XX(1)=-0.50000D0 
XX(2)=0.89000D0 
XX( 3 )=01 . 50000D0 
XX(4)=2.89Q00D0 
XX(5)=03 . 50000D0 
XX(6)=4.09000DO 

QAAAAAAAAAAxAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA 

C  INITIAL  VALUES  OF  T'S  FOR  CORRELATION  COEFFICIENT  GREATER  THAN  A1/A2 
TT( 1 )=-0 . 67000D0 
TT(2 )=0 .89000D0 
TT(3)=01 .67000DO 
TT (4) =2. 89000 DO 
TT( 5}=03 . 50000D0 
TT ( 6)=4 . 89000DO 

QAAAAAAAAAAxAAAAAA AAA AAA AAA AAA A A AA AAAAAAAAAAAAAAAAAAAAAAAAAAAAA 

C  INITIAL  VALUES  OF  T'S  FOR  CORRELATION  COEFFICIENT  LESS  THAN  A1/A2 
C  PUT  TT( 1 )  >  TT (2 ) > . >TT(N) 

AERR=0 . ODO 

RERR=0 . 00010D0 

DO  500  N=1 , 5 

DO  11  1=1,2 

R=DFLOAT ( 1-1 ) *0 . 250D0 

DO  20  M=1 , 20 

C=C1 (M) 

K=0 

5  K=K+1 
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V.V.v 


.w 


life, 


lvv;,v. 


5 


non 


WRITE(6 , 60)K,T1 ,T11 ,T2 ,T22 
IF  (K  .GT. 100)  GO  TO  10 
DO  25  L=1.N 
T(L)=TT(L) 

X(L)=XX(L) 

CONTINUE 


XX(1 )=(A1*A1/2.0DO+DLOG(C)+DLOG( DERFC ( ( T ( 1 ) - R*X ( 1 ) ) / 
1DSQRT ( ( 1 . 0D0-R**2 ) *2 . 0D0 ) ) 

1/DERFC(  (T(l)-A2-R't{X(l)-Al) )/ 


1/DERFC(  (T(l )-A2-R*{X(l )-Al 
lDS^RT ( ( 1 . 0D0-R**2 ) *2 . 0D0 ) ) 


IF  (N  .EQ.  1)  GO  TO  16 

A21=(S1*A2-R*S2*A1)/(S1*S2*DSQRT((1.0DO-R**2)*2.0DO 


A12=(S2*Al-R*Sl*A2)/{Sl*S2*DSgRT((1.0DG-R 
DO  15  P=1,N-1 

A=(S1*X(P)-R*S2*T(P))/(S1*S2*DSQRT((1.( 
B=(S2*X(P+l)-R*Sl*T(Pj )/ (S2*S1*DSQRT( (1 . 
TT(P)=(A2*A2/2 . ODQ+DLOG(C)+DLQG( (DERFC (A) - 


*DSQRT  ( ( 1 . 0DQ-R**2 )  *2 . 0D0 ) ) 

:i*S2*DSQRT ( ( 1 . 0D0-R**2)*2 .  0D0 ) 
S2*S1*DSQRT((  1 . 0D0-R**2)*2 .  ODQ  J 


B=(S2*T(P)-R*S1*X(P+1))/(S2*S1*DSQRT((1.QD0-R**2)*2.0D0)) 
XX(P+1 )=(Al*Al/2 . ODO+DLOG(C)+DLOG( (DeRFC(A)- 
1  DERFC \B) )/ (DERFC (A-A21 ) -DERFC (B-A21) ) ) )/Al 
CONTINUE 

CONTINUE 

TT(N)=(A2*A2/2.0DO+DLOG(C)+DLOG(DERFC( (X(N)-R*T(N) )/ 

1DSQRT( ( 1 . 0D0-R**2 ) *2 . 0D0 ) ) 

1/DERFC((X(N)-A1-R*(T(N)-A2))/ 

1DSQRT((1.0D0-R**2)*2.0D0)) 

1 ) )/A2 

IF( (DABS (T(M)-TT(N))  .GT.  0 . 10D-05) .OR. (DABS (X(N) -XX(N) )  .GT. 
1  0 . 10D-05 ) )GO  TO  5 

CONTINUE 
X(N+1 )=10 . 0D0 
APD=0 . 0D0 
APF=0.0D0 
DO  31  Q=1 ,N 
A=X(Q) 

B=X(Q+1 

X11=T(Q) 

APD=APD+Q . 50D0*DCADRE ( FI , A , B , AERR , RERR , ERROR , IRE1 ) 

IF  (IRE1  .NE.  0)  WRITE (6,60) IRE1 

APF=APF+0 . 50D0*DCADRE (F2 , A, B , AERR , RERR , ERROR , IRE1 ) 

IF  ( IRE1  .NE.  0)  WRITE(6 , 60 ) IRE1 
CONTINUE 


,(M,I,N)*i 
.(Mf I,N)*i 


R3 (M, I ,N)=1 .0D0+C1 (M)*PF1 (M, I ,N) -PD1 (M, I ,N) 
£************ ****************** ***************** ************ 
C  OUTPUT  SYSTEM  PARAMETERS 
DO  120  J=1  N 

WRITE(9 ,62)  X(J) ,T(J) 

120  CONTINUE 

WRITE (9 , 61 )C ,T( 1 ) ,  T(  2) ,APD,APF 
WRITE (8 ,61/APD ,APF 

WRITE(6,61)C,X(1) ,X(2) ,APD,APF,PDC,PFC 
20  CONTINUE 
11  CONTINUE 

C  WRITE (9, 90) 

WRITE (8, 90) 

500  CONTINUE 

C****** **************************** ************************* 

C  OUTPUT  AVERAGE  COST 
DO  501  J=1 , 2 
DO  502  1=1,20 

WRITE < 10 , 61 )C1 ( I ) , (  R3  ( I , J , N)  ,N=1,5) 

Q*********^f**^T*rr******^r**w****^*K**********  ***************** 

C  OUTPUT  PROB.  OF  DETECTION  AND  PRCB.  OF  FALSE  ALARM 
WRITE (8 , 61 ) (  PD1(I,J,N),PF1(I,J,N)  ,N=1,5) 

502  CONTINUE 


!  15 


WRITE<8,90) 

501  CONTINUE 

60  FORMAT ( 1X,I2 ,9(1X, F6 .4) } 

90  FORMAT ( 2X  ‘CON  B  ************************************ » \ 

61  FORMAT(lx!lO(lX,F6.4)) 

62  F0RMAT(1X,2(1X,F15 .8) ) 

STOP 

END 

FUNCTION  FI (X) 

REAL* 8  X, FI , A1 , A2 ,R,X11 , Fll , F12 

COMMON  Xll , R 

A1=4.0D0 

A2=2.0D0 

F11=DEXP(-(X-A1)**2/2.0D0)/ 

1DSQRT ( 3 . 0D0*DATAN ( 1 . 0D0 ) ) 

F12=DERFC{ (X11-A2-(X-A1)*R)/ 
1(DSQRT(2.0D0*(1.0D0-R**2)))) 

F1=F11*F12 

RETURN 

END 

FUNCTION  F2(X) 

REAL*8  X , F2 , R,X11 , Fll ,F12 

COMMON  Xll ,R 

FI 1=DEXF ( -X**2/2 . 0D0 ) / 

1DSQRT ( 8 . ODO*DATAM ( 1 . ODO ) ) 

F12=DERFC((X11-X*R)/ 

1DSQRT(2 .0D0*(1 .0D0-R**2) ) > 

F2=F11*F12 

RETURN 

END 


APPENDIX  D 

PROGRAM  LISTING  TO  CALCULATE  PARAMETERS  OF  THE  TWO 
QUANTIZERS  FOR  THE  CASE  OF  EXPONENTIAL  DISTRIBUTIONS 


Q  ************************* 

C  THIS  PROGRAM  CALCULATES  THE  OPTIMUM  N-LEVEL  QUANTIZERS  OF  TWO  SEN- 
C  SOR  OBSERVATIONS  OF  EXPONENTIAL  DISTRIBUTIONS 
C  TO  MINIMIZE  A  GLOBAL  SYSTEM  RISK  FOR  FUSION 
C  SEE  CHAPTER  IV 

Q  ************************* 

REAL*8  X(8),T(8),XX(8),TT(8),PD,PF,PD1(2G,9, 6),PF1(2Q,9,6) ,PD2 
1  ,PF2 ,PDC , PFC,C1 (20) , C,X2 , Xll . APD , APF , R3 (20 , 2 , o) , AL, BL,Y 

I, APC(20,2) ,AFC(20,2) ,R4(20,2) , AD2 , AF2 ,RR2 ,AP 
INTEGER  K,N , I , IER1 , IER2 ,M,P ,  Q ,  L  ,MX 

DATA  Cl/ 50 . 0D0 , 40 . ODO , 30 . 0D0 , 20 . ODO , 1 5 . 0D0 , 10 . 0D0 , 9 . 0D0 , 

18 . ODO , 7 . 0D0 , 6 . 0D0 , 5 . 0D0 , 4 . 0D0 , 3 . 0D0 , 2 . 0D0 , 1 . SODO , 

II.  ODO, .90D0 , . 80D0 . .70D0. .6000/ 

£  ************************* 

C  INITIAL  VALUES  OF  QUANTIZER  PARAMETERS 
C  FIRST  QUANTIZER 

XX(1 )=OO.OOOOODO 
XX(2)=0 .89000DO 
XX( 3 )=01 . 50000D0 
XX(4)=1 .89000DO 
XX( 5 )=02 . 50000D0 
XX(6 )=3 . Q900QD0 
C  SECOND  QUANTIZER 

TT(1)=3.090000D0 
TT(2)=2 . 50000D0 
TT (3) -01. 89000 DO 
TT(4)=1 . 50000D0 
TT(5 )=00 . 89000D0 
TT(6)=0.00000D0 
DO  500  N=3 , 6 
DO  11  1=1,2 

C  PARAMETERS  OF  THE  EXP.  DISTRIBUTIONS 
AL=DFLOAT(I)*Q.50D0 
DO  20  M=1 , 20 
C=C1 (M ) 

DO  55  P=1 , N 

TT(P)=DLOG(2.0DO*C)*DFLOAT(N-P+1)/DFLOAT(N)/AL 
XX(N-P+1 )=DLOG( 2 . 0D0*C ) *DFL0AT (N-P+l ) /DFLOAT(N) /AL 
55  CONTINUE 
K=0 

5  K=K+1 

DO  89  PP=1,N 

T (PP)=TT ( PP ) 

X(PP)=XX(PP) 

89  CONTINUE 
£************ ********* 

C  INPUT  MAXIMUM  NUMBER  OF  ITERATIONS 
MX=200 

IF  (K  . GT . MX )  GO  TO  10 
C  XX ( 1 )=DLOG( 2 . 0D0*C)/AL-TT ( 1 ) 

XX( 1 )=0 . ODO 

TT(l)=(DLCG(DEXP(-AL*XX(l))+DEXP(-AL*XX(2)))+DLOG(2.0D0*C))/AL 
DO  15  P=2.N-1 

XX(P)=(DLOG{DEXP(-AL*TT(P))+DEXP(-AL*TT(P-1)))+DLOG(2.0DO*C))/AL 
TT(P)=(DLOG(DEXP(-AL*XX(P) )+DEXP( -AL*XX(P+1 ) ) ) +DLOG ( 2 . ODO^C) )/AL 
15  CONTINUE 

XX(N)=(DLOG( 2 . ODO*C)+DLOG(DEXP (TT(N) )+DEXP ( -AL*TT(N-1 ) ) ) )/AL 
C  TT(N)=DLOG(2.0D0*C)/AL-XX(N) 

TT(N)=0 . ODO 
C  ACCURACY  CHECKING 
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•a 


a 


C  INPUT  PRECESSION  HERE 
AP=0 . 10d-05 

IF<(DABS(XX(N)-X(N))  .GT.  AP) .OR. (DABS(TT(l)-T(l) )  .GT. 
1  AP))GO  TO  5 


IF<(DABS(XX(N)-X(N))  .GT.  AP) .OR. (DABS(TT(l)-T(l) )  .GT. 

1  AP))GO  TO  5 

CONTINUE 
WRITE (8 . 60 )  K 
BL=2.0DO*AL 
APD=0 . ODO 
APF=0 . ODO 
DO  81  0=1, N-l 

APD=APD+DEXP ( - AL*TT ( Q ) ) * ( DEXP ( - AL*XX ( Q ) ) - DEXP ( - AL*XX ( Q+l ) ) ) 
APF=APF+DEXP ( -BL*TT (Q) ) * (DEXP ( -BL*XX(Q) ) -DEXP ( -BL*XX(Q+1 ) ) ) 
CONTINUE 

APD=APD+DEXP ( -  AL*X ( N ) ) 

APF=APF+DEXP(-BL*X(N) ) 

APC(M,I)=(1. ODO+DLOG( 4 . ODO*C) )/ (4 . 0D0*C) 

AFC(M, I )=( 1 . 0D0+DLGG( (4 . 0D0*C)**2) )/ (4 . QD0*C)**2 
PD1 (M, I ,N)=APD 
PF1(M, I ,N)=APF 

R3(M, I .N)=l .0D0+C1 (M)*PF1 (M. I ,N) -PD1 (M, I ,N) 

R4(M, I )=1 . 0D0+C1 (M) *AFC(M, I ) -APC (M, I ) 


DO  120  J=1 , N 

£**********xx*x:fc***xx  ■k'k'k-k'k-k’k'K'k'k 

C  OUTPUT  QUANTIZER  PARAMETERS 
WRITE (9 ,62)  XX(J) ,TT( J) 

120  CONTINUE 

WRITE (9, 90) 

20  CONTINUE 
11  CONTINUE 

WRITE (9, 90) 

C  WRITE (9,90) 

WRITE (8, 90) 

500  CONTINUE 

DO  501  J=1 , 2 

DO  502  1=1,20 

AP2=1 . ODO/ ( 2 . 0D0*C1 (I ) ) 

AF2=AP2:i'*2 

RR2=1 .  ODG-t-Cl  ( I )  *AF2-AP2 

C********x*xx**x*xxx*x*xx*x  **x* 

C  OUTPUT  DETECTION  COST 

WRITE (10,61)C1(I) , RR2 , (  R3(I,J,N)  ,N=3 , 6) ,R4(I , J) 

C**-**x******xx**x***xx*x*xK***jt 

C  OUTPUT  PROB.  OF  DETECTION  AND  PROB .  OF  FALSE  ALARM 

WRITE (8 , 61 ) AP2 , AF2 , (  PD1 ( I , J ,N) , PF1 ( I , J ,N)  , N=3 , 5 ) , APC( I , J) 
1 , AFC( I , J) 

502  CONTINUE 

WRITE (8, 90) 

501  CONTINUE 

60  FORMAT (1X,I6,9(1X,F6.4)) 

90  FORMAT  (  2X,  ‘CON  B  **x**k****************************** 1  J 

61  FORMAT(lx!lO(lX,F6.4)) 

62  FORMAT ( IX, 3 (IX, FI  5. 8)) 

STOP 

END 


%  k%  .N 


^  "*  ,*•  ,*•  .  *  _  *  kN  _ "*  _  *  ,  •  ,*•  *■  . *>  .*•  a  *  »'* >  * >  * 


APPENDIX  E 

PROGRAM  LISTING  TO  CALCULATE  PARAMETERS  OF  THE  TWO 
QUANTIZERS  FOR  EXAMPLE  3,  CHAPTER  IV 

C  THIS  PROGRAM  CALCULATES  THE  OPTIMUM  QUANTIZER  PARAMETERS  OF  TWO 
C  N-LEVEL  QUANTIZERS  IN  ORDER  TO  MINIMIZE  A  GLOBAL  SYSTEM  RISK  FOR 
C  DETECTION  OF  SIGNALS  WITH  DIFFERENT  VARIANCE.  N=3  ,  N=5. 
c  SI  =  SIGNAL  VARIANCE  UNDER  HI 
c  SO  =  SIGNAL  VARIANCE  UNDER  HO 
c  T1  =  QUANTIZATION  POINT  FOR  N=3  (  T1,-T1  ) 
c  XI  ,X2  QUANTIZATION  POINTS  FOR  N=5  (XI ,X2,-X1,-X2) 
c  C | (20 )  ARRAY  OF  RATIO  OF  COSTS 
c  K  =  Number  of  iterations. 

REAL*8  T1,T11,S1,S2,R,C,X2,PD,PF,C1(15) ,Z31 ,Z21 ,X3 ,X33 ,X22 
1 ,TS , SSS ,TTT (10) , XI , PD3 ,PF3 , PDC , PFC ,R2 , R3 ,RC 
INTEGER  K,N,K1 

DATA  Cl/01 . 0D0 , .900D0, .800D0, . 700D0 , 0 . 60D0 , 0 . 50D0 , 0 .40D0 , 

10 . 30D0 , 0 . 20D0 , 0 . 10D0  , .090D0, . 080D0 , . 070DO , 0 .060D0 , 0 . 050D0/ 
S1=1.QD0 
S0=DSQRT ( 2 . 0D0 ) 

c  INITIAL  VALUES  OF  THE  QUANTIZER  PARAMETERS 
Tl=01 . 15800D0 
Xl=01 . 15800D0 
X2=00 . 15800D0 
WRITE (6 , 60)K, SI ,S0 , C 
WRITE (9 , 60}K,S1,S0,C 
WRITE ( 8, 60 ;K, SI ,  SO ,  C 
SSS=(1.0D0/S0**2-1.0D0/S1**2)/2.0D0 
DC  101  N=2 , 2 
DO  100  1=1,15 
C=C1(I) 

K=0 

5  K=K+1 

IF  (K  . GT. 100)  GO  TO  10 

T11=T1 

TS=(DL0G(S1/S0)+ 

1DL0G(C)+DL0G( (DERF(T1/DS0RT(2 . 0D0 ) 

1/SO) )/(DERF(Tl/DSQRT(2.0D0)/Sl) ) )*DFL0AT(N-1))/SSS 
C  IF  (TS  .GT.  0.0D0)  T1=DSQRT(TS) 

T1=DSQRT (DSQRT (TS ) ) 

IF( (DABS(Tl-Tll)  .GT.  0.10D-05))  GO  TO  5 
CONTINUE 

10  TTT(N)=T11 
K1=0 

55  K1=K1+1 

IF  (IQ  . GT. 100)  GO  TO  15 

X33=X3 

X22=X2 

X2=(DL0G(S1/S0)+ 

1DL0G(C ) +DLOG ( (DERF ( X3/ DSQRT (2 . 0D0 ) 

1/SO ) ) / (DERF(X3/ DSQRT (2.0DQ)/S1))))/SSS 
C  IF  (TS  .GT.  O.ODO)  T1=DSQRT(TS ) 

X2=DS0RT(X2 ) 

Z30=DERF ( X3/ DSQRT (2 . 0D0 ) /SO ) 

Z20=DERF  X2/DSQRT  2.0D0  /SO 
Z31=DERF(X3/DSQRT  2.0D0  /SI 
Z21=DERF(X2/ DSQRT ( 2 . 0D0 ) /S 1 ) 

X3=(DLOG(S1/SO  )+• 

1DLCG ( C ) +DLQG( (Z30-Z20)/ (Z31-Z21 ) ) )/SSS 
X3=DSQRT ( DABS ( X3 ) ) 

IF( ( (DABS ( X2-X22 )  .GT.  0.10D-05))  ,OR.((  DABS(X3-X33)  .GT. 

10 . 10D-05 ) ) )  GO  TO  55 
CONTINUE 
TTT(N)=T11 
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TS=-DL0G(S0**2/S1**2/C)/SSS 

PDC=1 . ODO-DEXP(-TS/ (2 . 0D0*S1*S1 ) ) 

PFC=1.0DO-DEXP(-TS/(2.0DO*SO*SO)) 

RC=1.0D0+C*PFC-PDC 

WRITE (9 , 60)K,T1 ,T11 , X2 ,X22 ,X3 ,X33 

WRITE(6,60)K,T1,T11 


T1,T11 


PD3= ( DERF  v X22/DSORT (2 
1  *DERF(X33/DSQRT(2.0D0)/ 
PF3={DERF (X22/DSORT( 2 


RT(2.0D0)/S1)*2.0D0-DERF(X33/DSQRT(2.0D0)/S1) ) 
)DO)7si) 

RT (2 . 0D0 ) /SO ) *2 . ODO-DERF (X33/DSQRT ( 2 . ODO ) /SO ) ) 


WRITE (8 , 60 )N , PDC ,  PFC , PD , PF,PD3 ,PF3 
WRITE( 10 , 60 )N, C , R2 . R3 , RC 

IF  ((I  .EQ.  1)  .OR.  (I  .EQ.  10  ))  WRITE(10,60)N,C,R2,R3,RC 
WRITE(6,60)N,C,TTT(N) ,PD,PF,PD3,PF3 
CONTINUE 
CONTINUE 

FORMAT (1X,I3,6(1X,F10.7)) 

STOP 
END 


a 


APPENDIX  F 

LINEAR  MINIMUM  MEAN  SQUARE  ESTIMATE  OF  Y, 

Having  Y  =(  ylq  y2  f  the  LMMS  estimate  of  y,  and  the  corresponding  mean 
square  error  are  given  by  [34]: 


1 


Vj  =  E{yi  Z'JEtZZ1}*1  1 


\ 


E{e2  ]  =  Ely,2  )-E{y,  Yl )  E{Y  Yl  l'1  E{ Y  y,  } 


where 


1 


3 


E{y1Il}=[E{y,y  ]  E{y,  y2  }  ] 


{J  Jt  ]  Jfr2!, }  E<  yi,  y2  n 

[§{y2yiq}  E(y22)  J 


The  entries  of  these  matrices  are: 


E{yl  >'iq  )  “jlfi  QiE{>ri  'ytq  =Qj5 

-T  P:  Q.  C. 

jarj  )  VJ  i 
=  p  <T,2 

E^2  yiq  )  ~jXjPj  E^2  '  yiq  =  Qj  ^ 


E(y2  /yiq  *Qj  }“E{{y2'  yt  }/yiq  “Qj  > 


(F.7) 


For  the  case  where  and  y2  are  jointly  gaussian,  we  can  write 

E(y2  yiq  > = P  ffi  fj  Qj  cj/ffi2  (F-8) 

E^2  yiq  >  =  P  ^1  ^2  (F-9) 


E(yiq2)  (R1°) 

=  <*i2  n  ■ 

Inserting  these  in  (F.I  )  and  (F.2  )  and  performing  matrix  multiplications  yields 

>1  =  jo-  p2  )p ylq  +  p  (n  -p2  )y2J ;(n -p2 p2 )  (6-2) 

and 

E{e2  }  =ffl2  (1-p2  )(l-co  )/(l-p2  o  )  (6.11) 

where  <0  is  given  by: 

to  =  n2  ;r\.  (6.12) 
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APPENDIX  G 

SOLUTION  OF  THE  LLOYD-MAX  QUANTIZER  PARAMETERS  BY 
THE  METHOD  OF  SUCCESSIVE  SUBSTITUTION 

1.  INTRODUCTION 

The  minimum  distortion  quantizer  parameters  [18,23],  as  well  as  parameters 
based  on  other  criterion  such  as  quantizers  for  signal  detection  [41],  minimum  risk 
quantizers  and  quantizers  for  LMMS  estimation  ( rror  dealt  with  in  this  thesis,  can  be 
solved  by  Max's  trial  and  error  technique  [18].  There  are  also  many  other 
approximation  methods  to  calculate  the  quantizer  parameters  [42],  [43]  and  [44], 

In  this  Appendix  we  apply  the  method  of  successive  substitution  and  its 
modifications  [19]  to  solve  for  the  Lloyd-Max  quantizer  parameters.  It  is  more  accurate 
and  computationally  more  efficient  than  the  previously  reported  methods.  It  is  shown 
to  easily  generate  7  bit  (128  level)  optimum  quantization. 

2.  STATEMENT  OF  THE  PROBLEM 

The  Lloyd-Max  minimum  mean  square  distortion  quantizer  problem  deals  with 
transforming  a  random  variable  X  of  differentiable  probability  density  function  f[x) 
into  the  N-level  discrete  random  variable  Y. 


Y(X)-Y.  forXelx.  ,xi+1  ] 

The  optimum  parameters  minimize  the  distortion  D 

D  -  ?  J  X,+  \x-y.  )2  fix)  dx 
i*l  x. 


(G.l) 


(G.2) 


with 


.00  =Xj  S  X2  ^  ...<;  XN  ^  XN+  J-W 


Differentiating  D  with  respect  to  x;  and  y;  yields  the  following  necessary  conditions  of 
optimality  : 


Xj  =(Vj  +y.  +  1  )/2  ,i«2,3,...N 


(G.3) 
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f(x)dx  )  ,i=  1.2...N 


(G.4) 


y.  =(J  X,  +  1  x^x)dx)/(J  Xi  +  1 

xi  xi 

a  set  of  simultaneous  equations  of  propagating  character.  That  is,  if  y1  is  chosen 
correctly  then  x2  can  be  calculated  from  (G.4  ),  y2  from  (G.3  ),  x3  from  (G.4  )  and  so 
forth  [18].  In  this  case  the  value  of  yN  calculated  from  (G.3  )  must  agree  with  its  value 
calculated  from  (G.4  )  with  +  1  =  so  .  This  w’as  the  core  of  Max's  trial  and  error 
algorithm;  to  pick  a  value  for  yj  and  calculate  the  parameters  up  to  and  including  yN  , 
which  must  agree  with  the  value  of  yN  calculated  from  (G.4  ),  otherwise,  to  pick 


another  value  of  Vj.  Let  us  put  the  system  of  equations  in  the  form 

Z=£(Z)  (G.5) 

where  Z  is  a  2N-1  vector  given  by: 

-  -  i  x2  -  2 . -  N  ^  (G.6) 

and  apply  the  iterative  substitution 

Z„*»-ff(Zold)  (G.7) 

with  a  suitable  initial  guess.  The  convergence  is  guaranteed  if  d  Gfc  id  is  sufficiently 
small  for  even-  k,j=  1,2 . 2N-1  [19],  From  (G.4  ) 

3  Gj  :d  v.  =[(xj  +  1  -y.  )f (xj  +  1  )  +  (v.  .Xj  )f(xj  )]/(2P.  )  (G.8) 

where  R  is  the  probability  the  input  of  the  quantizer  is  in  the  interval. 

Pj  =f^i  +  1  ^x)dx-  (G.9) 


The  numerator  in  (G.8  )  is  an  approximation  of  the  integral  in  (G.9  )  by  the 
trapezoidal  rule  with  the  subdivision  (x.  ,v.  ,  xj  +  1  ],  so  the  value  of  the  derivative  is 
very  likely  less  than  one.  Also,  substituting  for  v.  and  y.  +  (  in  (G.3  )  from  (G.4  )  and 
differentiating  with  respect  to  Xj  it  is  easily  to  show  that 


(G.10) 


d  G.  Id  Xj  =  (v.  -Xj  )f(x.  )/(2P.)+(x.  -yH  )Rx.  )/2Pj  l 


which  is  less  than  ( 6  G.  Id  y.).  The  method  can  be  more  efTicient  if  we  use  the  updated 
values  in  the  same  iteration.  In  this  modification  of  the  method  the  best  current 
values  of  the  parameters  are  used.  This  choice  may  also  enhance  convergence.  The 
method  also  avoids  the  tedious  calculation  of  the  upper  limit  of  the  integral  to  solve  for 
the  next  x^  in  (G.4  ). 

3.  NUMERICAL  RESULTS 

We  have  solved  for  the  quantizer  parameters  for  a  gaussian  random  variable  of 
zero  mean  and  unit  variance  for  several  values  of  N  up  to  128.  Also  the  mean  square 
error  D  and  the  output  entropy  (-£k  Pk  log2  (Pk  ))  have  been  calculated.  The  results 
presented  in  Table  4  show  that  in  several  cases  Max's  results,  which  were  only 
available  up  to  N  =  36,  are  not  accurate  in  the  last  digit. 

Key  to  Table  4 

The  numbering  in  the  table  is  as  follows. 

1.  For  X  even,  each  table  begins  with  the  (N72  +  1)^  parameters.  In  this  case  the 
(N72  +  1)^  value  of  x  is  zero. 

2.  For  N  odd,  Each  table  begins  with  the  (N.'2  +  2)th  parameters.  In  this  case  the 
(N.2  +  2)^  value  ofy  is  zero. 

Negative  parameters  can  be  calculated  from  the  symmetry  relation 


xi  *  Vi*: 


(G.ll) 


and 


v. 

'i 


^n-j +  1 


(G.12) 


A  FORTRAN  program  to  calculate  the  parameters  .distortion  and  entropy 
follows  Table  4.  The  only  input  to  the  program  is  N,  the  number  of  quantization  levels. 


TABLE  4 

MAX'S  QUANTIZER  PARAMETERS  FOR  THE  NORMAL  DISTRIBUTION 


N  =  2  N  =  7 


J 

X 

Y 

J 

X 

Y 

2 

0. 000000 

0. 797885 

4 

-0. 280289 

0. 000000 

5 

0.  280288 

0. 560577 

ERROR  = 

0. 363380 

6 

0.  874362 

1.  188147 

ENTROPY  = 

1.  000000 

7 

1.  610758 

2. 033369 

N  =  3 

ERROR  = 

0. 044000 

J 

X 

Y 

ENTROPY  = 

2. 646931 

2 

-0. 612003 

0. 000000 

.  N 

=  8 

3 

0. 612003 

1. 224006 

— 

J 

X 

Y 

ERROR  = 

0. 190174 

•  _  — . 

ENTROPY  = 

1. 535789 

5 

0.  000000 

0. 245094 

6 

0.  500550 

0. 756005 

N 

=  4 

7 

1.  049957 

1.  343909 

8 

1.  747927 

2. 151946 

J 

X 

Y 

— 

-  -  -  - 

ERROR  = 

0.  034548 

3 

0. 000000 

0. 452780 

ENTROPY  = 

2. 824865 

4 

0. 981599 

1. 510418 

N 

=  9 

ERROR 

0. 117482 

ENTROPY  = 

1. 911099 

J 

X 

Y 

N 

=  5 

5 

-0. 221819 

0. 000000 

6 

0.  221819 

0. 443639 

J 

X 

Y 

7 

0.  681217 

0. 918796 

8 

1.  197594 

1. 476392 

3 

-0. 382284 

0. 000000 

9 

1. 865528 

2. 254664 

4 

0.  382284 

0.  764567 

“  “  —  - 

5 

1.  244357 

1.  724147 

ERROR  = 

0. 027853 

---- 

ENTROPY  = 

2. 982695 

ERROR  = 

0. 079941 

ENTROPY  = 

2. 202916 

N 

=  10 

N 

=  6 

J 

X 

Y 

J 

X 

Y 

6 

0. 000000 

0.  199623 

—  -  -  . 

7 

0. 4C4740 

0. 609857 

4 

0. 000000 

0. 317716 

8 

0. 833841 

1.  057825 

5 

0. 658911 

1. 000106 

9 

1. 324583 

1.  591340 

6 

1. 446350 

1. 893595 

10 

1.  968218 

2.  345096 

ERROR  = 

0.  057978 

ERROR 

0.  022937 

ENTROPY  = 

2.  442789 

ENTROPY  = 

3.  124584 

TABLE  4 

MAX'S  QUANTIZER  PARAMETERS  FOR  THE  NORMAL  DISTRIBUTION  (CONT'D.) 


N  =  11 


J 

X 

Y 

6 

-0. 183729 

0. 000000 

7 

0. 183729 

0. 367458 

8 

0. 559913 

0. 752367 

9 

0.  965597 

1. 178826 

10 

1. 435733 

1.  692639 

11 

2. 059193 

2. 425746 

ERROR  = 

0. 019220 

ENTROPY  = 

3.  253506 

N 

=  12 

J 

X 

Y 

7 

0. 000000 

0.  168438 

8 

0. 340142 

0.  511846 

9 

0.  694313 

0. 876779 

10 

1.  081245 

1.  285711 

11 

1.  534371 

1. 783030 

12 

2. 140733 

2.  498435 

ERROR 

0.  016340 

ENTROPY  = 

3.  371666 

-- 

=  13 

J 

X 

Y 

7 

-0. 156887 

0. 000000 

8 

0. 156887 

0. 313773 

9 

0. 476012 

0. 638251 

10 

0. 812600 

0. 986949 

11 

1. 184106 

1.  381263 

12 

1. 622890 

1.  864513 

13 

2. 214522 

2.  564525 

ERROR 

0.  014063 

ENTROPY  = 

3.  480744 

N  =  14 


J 

X 

Y 

8 

0. 000000 

0.  145706 

9 

0. 293513 

0.  441321 

10 

0. 595882 

0.  750443 

11 

0. 918039 

1.  085635 

12 

1. 276582 

1. 467528 

13 

1. 703070 

1. 938612 

14 

2. 281837 

2. 625062 

ERROR  = 

0. 012232 

ENTROPY  = 

3. 582050 

N 

=  15 

J 

X 

Y 

8 

-0. 136929 

0.  000000 

9 

0.  136928 

0.  273857 

10 

0. 414310 

0. 554764 

11 

0. 702949 

0. 851134 

12 

1.  013007 

1. 174879 

13 

1. 360468 

1. 546057 

14 

1. 776266 

2.  006474 

15 

2. 343670 

2.  680866 

ERROR  = 

0.  010737 

ENTROPY  = 

3. 676630 

"n 

=  16 

J 

X 

Y 

9 

0. 000000 

0. 128395 

10 

0. 258222 

0.  388048 

11 

0.  522404 

0. 656759 

12 

0.  799550 

0.  942340 

13 

1.  099286 

1.  256231 

14 

1. 437139 

1.  618046 

15 

1.  843532 

2.  069017 

16 

2. 400803 

2.  732590 

ERROR 

ENTROPY 


0. 009501 
3. 761328 


TABLE  4 

MAX'S  QUANTIZER  PARAMETERS  FOR  THE  NORMAL  DISTRIBUTION  (CONT'D.) 
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TABLE  4 

MAX'S  QUANTIZER  PARAMETERS  FOR  THE  NORMAL  DISTRIBUTION  (CON'T'D.) 
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TABLE  4 


MAX’S  QUANTIZER  PARAMETERS  FOR  THE  NORMAL  DISTRIBUTION  (CONT'D  ) 


-0. 083805 
0. 083805 
0.  252208 
0.  423045 
0 . 598128 
0. 779592 
0. 970115 
1. 173279 
1.  394213 
1. 640881 
1. 927050 
2. 280667 
2. 778634 


0. 000000 
0.  167610 
0.  336806 
0- 509283 
0. 686972 
0. 872212 
1. 068019 
1.  278540 
1. 509886 
1. 771876 
2. 082224 
2. 479110 
3. 078159 


ERROR  = 
ENTROPY  = 


0. 004041 
4.  384064 


JN 

=  z  / 

J 

X 

Y 

14 

-0. 077781 

0. 000000 

15 

0. 077780 

0.  155561 

16 

0. 233975 

0.  312389 

17 

0.  392106 

0. 471823 

18 

0. 553594 

0. 635364 

19 

0. 720073 

0. 804782 

20 

0. 893532 

0. 982281 

21 

1. 076518 

1. 170756 

22 

1. 272495 

1.  374235 

23 

1. 486469 

1. 598704 

24 

1. 726267 

1. 853829 

25 

2. 005461 

2. 157093 

26 

2. 351670 

2. 546247 

27 

2. 840977 

3. 135707 

ERROR  = 
ENTROPY  = 


0. 003483 
4.  491610 


0. 000000 
0. 161536 
0. 324498 
0. 490402 
0. 660961 
0. 838229 
1- 024813 
1.  224230 
1. 441544 
1. 684648 
1.  967207 
2. 316997 
2. 810502 


0. 080593 
0.  242480 
0. 406516 
0.  574288 
0. 747635 
0. 928823 
1. 120803 
1. 327657 
1. 555432 
1. 813865 
2. 120549 
2. 513445 
3. 107559 


ERROR  = 
ENTROPY  = 


0. 003746 
4. 438843 


15 

0.  000000 

0. 075012 

16 

0. 150307 

0.  225602 

17 

0. 301760 

0. 377919 

18 

0. 455569 

0.  533219 

19 

0. 613076 

0. 692934 

20 

0. 775854 

0.  858775 

21 

0. 945836 

1.  032897 

22 

1. 125522 

1.  218147 

23 

1.  318326 

1. 418505 

24 

1.  529205 

1.  639905 

25 

1. 765925 

1.  891945 

26 

2. 041975 

2.  192005 

27 

2.  384821 

2. 577637 

23 

2.  870169 

3.  162701 

ERROR  = 

0.  003246 

ENTROFY  = 

4.  542507 

TABLE  4 

MAX'S  QUANTIZER  PARAMETERS  FOR  THE  NORMAL  DISTRIBUTION  (CONT'D.) 


N 

=  29 

J 

X 

Y 

15 

-0. 072566 

0. 000000 

16 

0. 072566 

0. 145132 

17 

0. 218211 

0. 291291 

18 

0.  365424 

0. 439557 

19 

0. 515338 

0. 591119 

20 

0. 669236 

0. 747352 

21 

0.  828638 

0. 909923 

22 

0.  995431 

1. 080939 

23 

1.  172074 

1.  263209 

24 

1.  361940 

1.  460671 

25 

1-  569941 

1.  679211 

2  6 

1.  803788 

1.  928364 

2.  / 

2.  076890 

2. 225415 

28 

2.  416571 

2. 607727 

29 

2.  898177 

3. 188627 

ERROR 

0. 003032 

_ 

ENTROPY  = 

4.  591663 

N 

=  30 

J 

X 

Y 

16 

o. oocooo 

0. 070155 

17 

0. 140542 

0.  210928 

13 

0. 232019 

0.  353110 

19 

0. 425412 

0.  497714 

20 

0. 571795 

0.  645876 

21 

0. 722402 

0.  798927 

22 

0. 878709 

0.  958490 

23 

24 

1. 042565 

1. 216393 

1.  126640 

1.  306147 

25 

1. 403530 

1.  500912 

2  6 

1 . 608845 

1.  716779 

27 

1.  840001 

1.  963224 

28 

2. 110332 

2. 257440 

29 

2. 447027 

2. 636614 

30 

2. 925088 

3. 213562 

ERROR  = 

0. 002839 

ENTROPY  = 

4.  639193 

N  =  31 


J 

X 

Y 

16 

-0. 068008 

0. 000000 

17 

0. 068008 

0. 136016 

18 

0. 204446 

0. 272876 

19 

0.  342170 

0. 411464 

20 

0. 482101 

0. 552739 

21 

0. 625267 

0. 697794 

22 

0. 772858 

0. 847921 

23 

0. 926315 

1. 004710 

24 

1. 087456 

1. 170202 

25 

1.  253670 

1.  347138 

26 

1.  443261 

1. 539385 

27 

1.  646065' 

1. 752745 

28 

1.  874694 

1.  996643 

29 

2. 142413 

2. 288184 

30 

2. 476285 

2. 664385 

31 

2. 950981 

3. 237577 

ERROR  = 

0. 002664 

ENTROPY  = 

4.  685201 

N 

=  32 

J 

X 

Y 

17"“ 

0. 000000 

0. 065890 

18 

0. 131971 

0. 198052 

19 

0.  264715 

0. 331378 

20 

0. 399039 

0. 466699 

21 

0. 535816 

0. 604934 

22 

0. 676035 

0. 747136 

23 

0. 320850 

0. 894565 

24 

0. 971674 

1. 048783 

25 

1.  130294 

1.  211804 

26 

1. 299072 

1. 386340 

27 

1.  481284 

1. 576228 

28 

1.  681731 

1. 787233 

29 

1. 907981 

2. 028728 

30 

2.  173234 

2. 317739 

31 

2. 504429 

2. 691120 

32 

2. 975926 

3. 260732 

ERROR 

0. 002505 

ENTROPY  = 

4.  729784 

3 


TABLE  4 

MAX  S  QUANTIZER  PARAMETERS  FOR  THE  NORMAL  DISTRIBUTION  (CONT  D  ) 


N  =  64 


J 

X 

Y 

33" 

0.  000000 

0. 033409 

34 

0.  066844 

0. 100278 

35 

0.  133787 

0. 167297 

36 

0.  200932 

0. 234567 

37 

0.  268380 

0. 302193 

38 

0.  336238 

0.  370283 

39 

0.  404616 

0.  438950 

40 

0.  473632 

0.  508314 

41 

0.  543408 

0.  57-8503 

42 

0.  614079 

0.  649655 

43 

0.  685789 

0.  721922 

44 

0.  758695 

0.  795468 

45 

0.  832972 

0.  870476 

46 

0.  908816 

0.  947155 

47 

0.  986446 

1.  025736 

48 

1.  066112 

1.  106488 

49 

1.  148104 

1.  189720 

50 

1.  232757 

1.  275794 

51 

1.  320468 

1.  365141 

52 

1.  411709 

1.  458276 

53 

1.  507054 

1.  555831 

54 

1.  607210 

1.  658589 

55 

1.  713065 

1.  767542 

56 

1.  825759 

1.  883977 

57 

1.  946794 

2.  009611 

58 

2.  078211 

2.  146810 

59 

2.  222896 

2.  298981 

60 

2. 385143 

2.  471305 

61 

2.  571789 

2.  672274 

62 

2.  794840 

2.  917407 

63 

3.  078922 

3.  240437 

64 

3.  492269 

3.  744101 

ERROR  = 

0.  000644 

ENTROPY  = 

5. 710078 

TABLE  4 

MAX'S  QUANTIZER  PARAMETERS  FOR  THE  NORMAL  DISTRIBUTION  (CONTD.) 


-0. OOCOC1 
0.  033659 
0.  067331 
0.  101029 
0.  134765 
0.  168552 
0.  202404 
0  236333 
0.  2'0353 
0  304477 
0.  338720 
0  373097 
0  407620 
0.  442307 
0.  477172 
0.  51223  1 
0. 547502 
0.  583001 
0.  618747 
0. 654759 
0.  691055 
0. 727658 
0.  764589 
0.  80 187  1 
0. 839528 
0.  877587 
0.  916074 
0.  955019 
0.  994453 
1. 034409 

1.  074924 

1.  116037 
1.  157788 
1.  200223 
1.  243391 
1.  287346 
1.  332  146 
1.  37-057 
1.  424548 
1.  472299 
1.  521196 
1. 571339 
1.  622833 

1.  676905 

1.  730294 
1.  7?6'59 
1.  845085 
1.  905583 

1.  966503 
2. 034136 
2. 102032 

2. 175210 
2. 251195 
2. 321996 

2 .  ( 1 22 >2 
2  '  1 .  1 2  5 
2  611463 
2  0.1  '00 

2  84!:  99 

2.  9931  16 

3  14  553 } 

3  33'.-b3 1 

3.  591133 

3.  '962315 


0.  016828 
0.  050490 
0  084172 
7.  117836 
0.  151644 
0.  185460 
0.  219347 
0  253313 
0  287387 
0.  321567 
0.  355874 
0.  390320 
0.  424921 
0. 459693 
0. 494651 
0.  529912 
0.  505192 
0.  600810 
0.  63  6684 
0.  672833 
0.  709277 
0.  746039 
0.  783139 
0  820602 
0.  858454 
0. 896719 
0.  935428 
0.  974609 
1. 014296 
1-  054523 
1.  095326 
1.  136747 
1.  178828 
1.  221617 
1.  2d5  165 
1.  309527 
1.  354766 
1.  400948 
1.  448148 
1. 496450 
1.  5459  43 
1.  596733 
1  649933 
1.  ■'.’2677 

1  ’tov 
l'.  81540* 

1.  974752 

1.  936404 

2.  COC6U1 
2.  06  *67  1 
2.  137993 
2.  2120.77 

2  2°^ •  3"J 

2.  373':.  4 
2.  »'2ai  : 


*■  '  *  j  i  i 

2.  9C90  47 

3.  0«724v 
3.  231333 
3  4  4743  0 

3.  34934 

4.  189o94 


=  9  Q00153 

EIUF.CF2  =  6.  699583 


.Vv-v'AlV 


4.  PROGRAM  LISTING  TO  CALCULATE  THE  LLOYD-MAX  QUANTIZER 
PARAMETERS 

C  THIS  PROGRAM  CALCULATES  LLOYD-MAX  QUANTIZER  PARAMETERS  BY  THE  METHOD 
C  OF  SUCCESSIVE  SUBSTITUTION  FOR  THE  NORMAL  DISTRIBUTION  OF  ZERO  MEAN 
C  AND  UNIT  VARIANCE 
C  The  INPUT  TO  THE  PROGRAM  IS 

C  (1)  THE  NUMBER  OF  QUANTIZATION  LEVELS  N 

C  (2)  THE  MAXIMUM  NUMBER  OF  ITERATIONS  M 

C  (2)  THE  ACCURACY  AP 

Q  kkkkkkkkkkkkkkkkkkkkkkkkkkkkkk 

REAL*8  X(199),T(199),XX(199),TT(199),C  , DELTA, AP(199) ,AP 
1 , ERROR , ENTROP 
INTEGER  K(N,I,P,N1,N2,N3,M 
C=DSQRT(00 . 50D0/DATAN ( 1 . 0D0) ) 

C  DO  99  N=110 . 110 

Q  k  k  *  *  k  -k  k  -k  k  k  n  k  *  k  k  k  k  k  k  k  *  k  kkk  k  k  k  k  k  k  k  A*  *  *  *  k  *  ★  *  k  **  ifc  *  *  kkkk  k  **  A  A  k  *  * 

C  INPUT  THE  NUMBER  OF  QUANTIZATION  LEVELS 
N=100 

WRITE (9 , 65  )N 
WRITE (9, 66) 

WRITE  9,67) 
write(9.66) 

Qkkkkkkkkkyckkknkk-xkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk 

C  INITIALIZATION  OF  THE  QUANTIZER  PARAMETERS 
C 

DELTA=0.0150D0*DFL0AT(N) 

XX( 1 )=-10 . 50000D0 

TT(1)=-5.50000DO 

X(1)=XX(1) 

T ( 1 j=TT ( 1 ) 

DO  50  L=2  N 

TT(L)=TT(L-1) -DELTA 
XX(L)=(TT(I)+TT(I-1))/2.0D0 
X(L)=XX?L ) 

T(L)=TT(L) 

50  CONTINUE 

Qkkkkkkkkkkkkkkkkkkkkkkkk  kkkk k kkkk kkkkkkkkkkkkkkkkkkk 

C  BEGINING  OF  THE  ITERATIONS 
c  M  =  MAXIMUM  NUMBER  OF  ITERATIONS 
M  =  1050 
K=0 

5  K=K+1 

IF  (K  .GT.  M)  GO  TO  10 
IF  (K  .GT.  1)X(N)=XX(N) 

IF  (K  .GT.  1  T(N  =TT(N 

TT(1)=-C*DEXP(-XX(2)*XX(2)/2.0D0)/(DERFC(-10.0D0)- 
1DERFC(XX(2)/DSQRT(2 . 0D0) ) ; 

T ( 1 )=TT ( 1 ) 

IF  (  N  .EQ.  2  )  GO  TO  17 
DO  15  P=2.N-1 

XX(P)=(T(P)+T(P-1))/2.0D0 

X(P)=XX(P) 

TT(P)=DEXP(-X(P)*X(P)/2.0DO)-DEXP(-X(P+l)*X(P+l)/2.0D0) 
TT(P)=TT(P)*C/(DERFC(X(P)/DSQRT(2.0D0))-DERFC(X(P+1)/DSQRT(2.0 
1  0D0))) 

T(P)=TT(P ) 

15  CONTINUE 

17  CONTINUE 

XX(N)=(TT(N)+T(N-1))/2.0DO 

TT(N)=DEXP( -XX(N) *XX(N)/2 . 0D0 ) *C/DERFC(XX(N)/DSQRT(2 .0D0) ) 

X(N)=XX(N) 

T(N)=TT(N) 

N2=IDINT(DFLOAT( (N+2)/2) ) 

Nl=IDINT(DFLOAT((N+l)/2) ) 

QkkkkkknkkkkkkkkkkkkkkkkkkkirKk-xkkkkkkkkkkkkkkkkkkkkkkk 

C  CHECKING  THE  PRECISION  OF  THE  SOLUTION 
c  AP  =  REQUIRED  ACCURACY 
AP=0 . luD-6 


((M0D(N+1,2) 

CONTINUE 


EQ.  0) .AND . (DABS(X(N2) ) .GT.  AP))GO  TO  5 
.EQ.  0) .AND.  (DABS(T(N1)) .GT.  AP))GO  TO  5 


10  CONTINUE 

£********************** £**£&£* A************************ 

C  OUTPUT  RESULTS 

IF  (MOD(N, 2)  .EQ.  0)N3=N2 
IF  (M0D(N+1 ,2)  .EQ.  0)N3=N1 
WRITE(6,60)  K 
DO  120  J=1,N3 

IF  (J  .EQ.  1) 

1  WRITE (9,71 )J,  T(J) 

IF  (J  .GT.  1) 

1  WRITE (9 , 61 ) J,  X( J) ,T( J) 

120  CONTINUE 

X(N+1)=10.0D0 
X(1)=-10.0D0 
ERROR=0 . ODO 
ENTR0P=0 . ODO 
DO  222  1=1, N 

AP ( I ) =DERFC (X ( I ) /DSQRT ( 2 . ODO ) ) -DERFC ( X ( 
AP  ( I ) =AP ( I ) / 2 . ODO 


AP ( I )=DERFC (X ( I ) /DSQRT ( 2 . ODO ) ) -DERFC (X( 1+1 ) /DSQRT ( 2 . ODO ) ) 
AP ( I )=AP( I ) / 2 .ODO 
ERROR=ERROR+AP ( I ) *T ( I ) **2 

ENTROP=ENTROP-AP ( I ) *DLOG( AP ( I ) ) /DLOG( 2 . ODO ) 


CONTINUE 
ERR0R=1 . ODO 
WRITE (9, 66) 
WRITE(9-,62 
WRITE(9,66) 
WRITE (9,63) 
WRITE  9,66 
WRITE (6,72) 
WRITE (9, 90) 


399 

CONTINUE 

65 

FORMAT ( 

66 

FORMAT ( 

67 

FORMAT ( 

60 

FORMAT ( 

90 

FORMAT ( 

61 

FORMAT ( 

71 

FORMAT ( 

62 

FORMAT ( 

72 

FORMAT ( 

63 

FORMAT ( 

-ERROR 

ERROR 

ENTROP 


=  M7) 


14,  2(2X,F9.6) ) 

14 , 11X,  2(2X,F9 .6) ) 
'ERROR  =' ,  2 ( IX, F9 .6) ) 
'#  ITERATIONS  =  1 ,17  ) 
'ENTROPY  =' ,  2 ( 1X,F9 . 6) ) 
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